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FOREWORD 



The challenge to assist youth in gaining the skills and knowledge 
to plan for and acquire meaningful careers is a major concern 
of the educational community, particularly high schools. In 
response to this challenge, The National Center developed and 
tested, under sponsorship of the National Institute of Education 
(NIE) , the Career Planning Support System (CPSS) . CPSS is a set 
of staff instructional materials, that show high school staffs 
how to deliver improved, cost--ef f ective career guidance services 
that meet student needs and fall within the resources of the 
individual school . 

This report documents a controlled evaluation of the CPSS conducted 
between June 1, 1978 and November 30, 1979. By using ten experi- 
mental (used CPSS) and eight control (did not use CPSS) high 
schools. National Center staff conducted a national assessment 
of the effectiveness of the CPSS. The results of the national 
assessment shov/ that the career development programs in the ten 
high schools that used CPSS were significantly better than the 
eight schools xr^hich did not use the CPSS. We are pleased to 
report that, in a controlled evaluation, CPSS works. 

We are deeply grateful to the staff in the high schools and 
school districts who participated in the assessment study. Their 
cooperation and consistent enthusiasm were central to the success- 
ful completion of the project. The names of the staff, schools, 
and school districts involved in the national assessment are 
listed in Appendix L of this report. 

Additional thanks go to the project review panel who were required 
to rate anonymou£>ly the career development programs of the high 
schools participating in the assessment. The panel consisted of 
fifteen experts in the fields of career education practice and 
research. Their names are listed in Appendix G of the report* 

Special thanks go to Robert I. Wise, NIE Project Officer, who 
provided support and guidance during the CPSS evaluation , to 
Dr. Paul Raffeld, instrument consultant, for his advice in revising 
the rating instrument, and to Dr. James Altschuld, consultant. 
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Finally, we are grateful to the project staff who planned and 

conducted the assessment of CPSS, Mr- James A. Pearsol/ 

Dr, Ann R. Nunez, Mr. Vernon Padgett , Dr. Donald C. Findlay, 

Mr. Drew Denton, Ms. Susan Klaiber, and Dr. H. Lawrence Hotchkiss, 

Project Director. Special thanks go to Ms. Nancy Robinson, ^ 

Ms. Debbie Frederick, and Ms. Debbie Cantan for their help in 

preparing for and typing the final report. 



Robert Taylor 
Executive Director 
The National Center for Research 
in Vocational Education 
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ABSTRACT 

The National Center for Research in Vocational Education 
conducted the "Assessment of the Career Planning Support System" 
CO report information about the impact of the Career Planning 
Support System (CPSS) on high school staffs' planning and career 
development activities for^ student career development. 

CPSS is a package of materials designed to enable a high 
school staff to improve the effectiveness of its career develop- 
ment program through systematic program planning. 

The study used a pre-post, experimental and control group 
research design. Ten high schools were assigned to the experi- 
mental condition, using CPSS, and eight high schools were assigned 
to the control condition and did not use CPSS. The high schools 
were located in eight different school districts in Arizona, 
Colorado^ Florida, Illinois, Kentucky, Maryland, and Tennessee. 

Each school provided a part-time coordinator who was respon- 
sible for the preparation and completion of data collection forms 
and served as the on--site contact person with National Center 
staff. In the experimental schools, this contact person also 
served as the CPSS coordinator and chaired a CPSS steering com- 
mittee in the school. 

Two levels of measurement were used in the study. The most 
important, or primary, level involved the use of two data collection 
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forms and a rating instrument* The data collection forms were 
completed at each school and were used to collect infomation 
about each school's career development program. The rating instru-- 
ment was used by an external panel of experts to rate the informa- 
tion collected from each school on the datci collection fornis, . 
both pre and posttest. The fifteen panel members represented pro- 
fessional fields related to career development research and prac- 
tice. The instrumentation developed for the primary level of 
measurement was used to assess two major hypotheses: I- experi-^ 
mental school staffs, using CPSS , would demonstrate greater change 
toward the installation of a systematic plan for career guidance 
than would control schools, and Il-experimental school staffs 
would develop a plan that contains higher quality career develop- 
ment activities to improve student career development skills than 
would control schools. 

A secondary level of investigation involved assessments of 
CPSS steering committee members in the experimental schools. A 
test of CPSS knowledge and an attitude scale related to systematic 
program planning were administered to these school staff. The 
instruments were used to assess CPSS steering committee members' 
knowledge of CPSS and attitudes toward planning for career devel- 
opment programs both pre and posttest. This information was used 
to support Hypothesis I above. 

The findings reported for the primary level of measurement. 
Hypothesis I, were highly significant. The experimental schools 
displayed greater change toward the installation of a systematic 
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plan for career guidance than did control schools. The findings 
for the primary level of measurement. Hypothesis II, were mixed; 
however, a positive trend was observed in favor of the experimental 
schools regarding the quality of career development activities. 
The secondary level of measurement produced significant findings 
related to increased knowledge of CPSS and favorable attitudes 
toward systematic planning for career development programs among 
CPSS steering coirauittee members in the experimental schools • 

A major product of the study was the preparation of a report 
for the Joint OE-NIE Dissemination Review Panel- The Panel reviews 
the products of federally sponsored research for dissemination funds • 
The results of the Panel's evaluation of the report were not avail- 
able at the time the final report was published. 
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INTRODUCTION 

The National Center for Research in Vocational Education 
conducted the Career Planning Support System (CPSS) Assessment 
Study to report information about the impact of CPSS on high 
school staffs' planning and programming for student career 
development. The eighteen month study (June 1, 1978 - 
November 30, 1979) assessed the effects of CPSS in selected high 
schools during one academic year (October, 1978 - June, 1979). 
Project staff used the June - September, 1978 months to conduct 
site selection and instrument development activities, and the 
July - November, 197 9 months to complete ratings of school data, 
data analysis, and final reports. 

This controlled assessment involved the use of ten experi-- 
mental (use CPSS) high schools and eight control (did not use 
CPSS) high schools in eight different school districts throughout 
the United States. The results of the assessment are presented 
in this final report and were submitted in a condensed report to 
the Joint Dissemination Review Panel. (JDRP) of the U.S. Office 
of Education and the National Institute of Education for review 
and approval. 
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Description of CPSS 
The Career Planning Support System (CPSS) is a package of 
materials designed to enable a high school staff to improve the 
effectiveness of its career development program through system- 
atic program planning. "Career development" in the CPSS perspec- 
tive is defined as the process by which an individual student 
acquires the basic, non-technical skills to cope in the 
world of work. CPSS is a support system that helps organize a 
school's staff and resources to meet the career development needs 
of students; therefore, CPSS is not a package of materials that 
an individual student might use to explore careers. CPSS is^ a 
set of program management procedures that are designed to promote 
five basic student career development skill areas: 1) Personal- 
Social Awareness Skills, 2) Career Exploration Skills, 3) Job 
Acquisition Skills, 4) Education and Training Exploration Skills, 
and 5) Education and Training Acquisition Skills. Over time, 
student skills development in these five areas is the expected 
outcome of the procedures suggested by CPSS. 

CPSS is implemented by school staff and employs a package 
of handbooks, reproducible forms and filmstrips that guide the 
planning, implementation and evaluation of a high school's career 
development program. The following list describes a complete 
set of CPSS materials: 

« The Coordinator's Training Guide is a self -instructional 
training guide for the part-time CPSS coordinator. 

« The Coordinator's Handbook contains instructions that 
describe step-by-step procedures for managing and imple- 
menting CPSS in the high school. 



m Camera- Ready Forms are reproducible copies of each form 
needed for the questionnaires, instructions, CPSS Program 
Information File, etc, 

• Handbooks 

The Advisory Committee Handbook defines the responsi- 
bilities and duties of Advisory Committee members (five 
copies ) . 

Assessing Resources guides a resource leader in direct- 
ing a task force to collect information on and account 
for the use of resources in the school and community. 

Assessing Needs : Surveying provides instruction for 
preparing, administering, and collecting survey ques- 
tionnaires for students, graduates, parents, and faculty/ 
staff (five copies) . 

As sessing Needs: Tabulation contains instruction on 
manually tabulating data collected by questionnaires 
(five copies) . 

Analyzing Methods directs the methods specialist to 
report the availability of guidance methods and instructs 
him/her on how to integrate this knowledge into the 
construction and review of career development units. 

The Ma nual for Writing Behavioral Objectives is a 
self-instructional guide for the behavioral objectives 
specialist. 

Writing Behavioral Qbj ectives informs the behavioral 
objectives specialist about the function of behavioral 
objectives in the construction of career development 
units . 

Produc ing CPUs (Career Development Units) provides direc- 
tion for developing career guidance/development activities. 

• Fi Imstrip/Audio Tape Presentations include; 

AV-1: "An Orientation to CPSS" — designed to orient 
interested persons or special groups to CPSS. 

AV-2: "Shaping Program Goals"--an overview of how the 
needs and resources assessments lead to goals 
for your school . 

AV-3: "Behavioral Objectives" — used with the behavioral 
objectives manual. 
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AV-4 : "Producing CDUs" — an overview of the career 
development unit process • 

To accomplish the planning, implementation and evaluation 
procedures, CPSS recommends that a school coordinator lead the 
CPSS effort with the assistance of a working steering committee 
comprised of students, teachers, counselors, and administrators. 
The CPSS coordinator gives direction to the CPSS effort and 
chairs the steering committee; however, much of the planning, 
implementation and evaluation is performed by the steering com- 
mittee with the help of other school faculty and members of the 
community. 

The key procedural steps prescribed by CPSS are: 

1. Organization of school staff ; The CPSS coordinator, 
using CPSS manuals as guidelines, organizes a steering 
committee of faculty, students, counselors and adminis- 
trators who meet regularly to complete the following 
steps. For each of the steps below, CPSS provides com- 
plete instructions . 

2. Assessment of resources : Steering committee members 
compile a record of resources available in the school 
and community that might be used in the career develop- 
ment program. Resources include equipment, space, per-- 
sonnel, funds, and materials. Resource assessment also 
includes collecting demographic data describing the school 
and community, information about current career guidance 
and career development instructional activities in the 
state, district, school, and feeder schools, and a record 
of resources expended. The CPSS package includes a list 
of important categories of resources and tabulation 
forms for recording the resources in each major category. 

3. Assessment of student needs : Steering committee members 
with the help of other faculty and students survey 
students, faculty, recent high school graduates and 
parents in order to determine the career development 
needs of the student body. Camera-ready master copies 
of the necessary survey instruments are provided in 

the CPSS materials. 
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4. Wr iting cjoals and behaviora l objectives : Steering 
committee members use the results of the needs assess- 
ment to specify career development goals derived from 
needs and to list the goals in order of priority. 
Priorities depend, in part, on available resources. 
After the goals have been listed in order of priority, 
the highest priority goals are translated into behavioral 
objectives for students. 

5. Creating career development units (CPUs) : Interested 
faculty, with guidance from steering committee members, 
design CDUs. A CDU is a sequence of activities that 

is designed to achieve related sets of behavioral objec- 
tives for students. CPSS manuals prescribe the components 
of successful CDUs. The CDUs are geared toward the 
five basic career development skill areas listed earlier 
in this section. An example of a CDU might be a career 
development course or a sequence of related field trips. 

Annual program review : At the end of each year the 
steering committee members' review the CPSS career 
development program, including CDU development and 
implementation, the status of career development resources, 
progress toward program goals in satisfying student needs, 
and plan for the following year's efforts. 

7. Program reassessment : CPSS recoimnends that the needs 
assessment be administered every three years. After 
three years, the data from recent graduates, who were 
introduced to CPSS-derived activities in the previous 
three years, are compared with data from current students 
to determine if the program plans initiated through CPSS 
are working and if student needs in high priority areas 
have been diminished {over the threa year period) . 

The seven points listed above are the primary procedural 

components of the CPSS. The seven points are systematically 

interdependent in that steps one through four must occur first 

and lead to steps five, six, and seven. Approximately three 

academic years are required to install CPSS fully. The first 

year is devoted mostly to initial, systematic career development 

program planning, basically steps one through four above. After 

the first year's foundation is laid, the CPSS steering committee 
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focuses on the production and implementation of CDUs (step five) , 
the heart of the school *s career development program. This 
Assessment Study focused primarily on steps one through four; 
however/ some data related to step five were collected , 

CPSS, as a support system, is designed to provide the or- 
ganizational framework and procedural steps required to install 
an accountable, school-wide career development program. It does 
not prescribe what specific career development activities a school 
should use, but rather provides a means for a school to focus 
its career development program on the unique needs of its own 
students and within the bounds of its own resources. 

Overview of the Study 

Design 

The CPSS Assessment Study was conducted to assess the ef- 
fectiveness of CPSS. The study used a pre-^post, experimental 
and control group research design to determine the impact of 
CPSS on selected high schools during one academic year. Eighteen 
comparable high schools participated in the study. Ten high 
schools were assigned to the experimental condition, implem.enting 
CPSS, and eight high schools were assigned to the control condition. 
The following chart shows the months when the pre and posttest 
data were collected and when the CPSS treatment occurred. 
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Figure 



CPSS Assessment Study Research Design. 



Experimental 
Schools (10) 



1978 



1979 



Feb 


Mar 


Apr 


May 



PRE- 



CPSS 



POST — > 



Control 
Schools (6) 

(2) 



PRE- 



PRE* 



POST- 
POST- 



* Two control schools were unable to complete the 
pretest until 12/15/78. 



Obj ectives 

Two objectives guided the study. The objectives addressed 

the capacity of CPSS to produce 1) a systematic plan for career 

guidance and 2) career development activities that have a high 

probability of improving student career development skills. The 

objectives for the study are listed below: 

1. School counselors, teachers, administrators, and students 
involved in CPSS for one academic year will develop 
a systematic plan for career guidance as judged by an 
external panel of reviewers. 

a. CPSS steering committees in the experimental schools 
will demonstrate a working knowledge of the operation 
of the Career Planning Support System. 

b. Attitudes of CPSS steering committee members in 
the experimental schools toward systematic planning 
for career development programs will either be 

more favorable by the end of the school year or will 
not be any less favorable than before the initiation 
of CPSS. 
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2. School counselors, teachers, administrators; and students, 
involved in CPSS for one academic year will develop a 
plan that contains activities having a high probability 
of improving student career development skills as judged 
by an external panel of reviewers. 

Objective One is based on the assumption that "a systematic 
plan" is derived from coherently-related planning components. 
For the purposes of this study "a systematic plan for career guid- 
ance" consists of the following planning elements: 

• An organiz ational structure facilitating a career develop- 
ment program, to include clearly designated leadership , 
permanent active committees and work groups, and adminis - 
trative cooperation 

• An assessment of the career development needs of local 
students and use of the results of the needs assessment 
in the career development program 

• Creation of explicit career development goals reflecting 
assessed student career development needs 

• Creation of behavioral objectives designed to implement- 
the goals 

• Creation of student activities to achieve the objectives 
and goals 

Descriptive information related to the elements above was 
collected from both experimental and control schools, pre and 
posttest, and rated by expert raters to assess Objective One. 
It was expected that experimental schools would produce greater 
change toward a systematic plan for career guidance than would 
control schools. 

Objectives la and lb assume that demonstrated knowledge of 
CPSS procedures and favorable attitudes toward systematic planning 
will insure that experimental schools know how to develop a system- 
atic career development program plan and display favorable attitudes 



8 



19 



already or will develop favorable attitudes about systematic 
planning efforts. Demonstrated knowledge of CPSS and supportive 
attitudes lend credence to the expectation that the systematic 
planning steps developed in Objective One can be implemented. 

Objective Two is based upon the assumption that systematic 
career development program planning leads to higher quality 
activities to meet student needs for career development skills 
than activities found in schools that do not use systematic 
planning procedures. Although the one-year assessment period did 
not permit full implementation of CPSS and full development of 
CDUs, some data were collected allowing comparisons between experi- 
mental and control school career development activities. 

Measurement 

Two types of instrumentation were developed to meet Objectives 
One and Two. One type of instrumentation was the two data sources 
which were used to report the status of career development program 
planning efforts and career development activities in experimental 
and control schools both pre and posttest. The first data source 
was a Career Development Program Status Report (CDPSR) and the 
other was a Verification Checklist {VCLl . 

The second type of instrumentation was the Career Development 
Program Rating Instrument (HI) used by an expert panel of raters. 
The raters completed the rating instrument using the CDPSR and 
VCL as data sources. The purpose of the ratings (provided on 
the rating instrument) was to assess the presence or absence of 



key career development program planning components and the quality 
of career development program activities. 

The subset of objectives of Objective One, la and lb, were 
assessed through a separate set of instrumentation. A knowledge 
test of CPSS procedures. The Career Planning Support System 
Knowledge Test , was developed by project staff to assess steering 
committee members' knowledge of CPSS. An attitude scale. Percep - 
tions of Program Planning for Career Development , developed by 
project staff was designed to assess steering committee members' 
attitudes toward systematic planning. The data sources, rating 
instruments, knowledge test and attitude scale are described 
fully in the METHODOLOGY section. 

Table 1 displays the data sources and timing for the CPSS 
Assessment Study relative to the objectives of the study. 

Details about the progress of the Assessment Study and 
subsequent results follow. Site selection, instrument develop- 
ment, site monitoring, pre-post data collection, and the conduct 
of the expert panel of raters' meetings are all described in the 
METHODOLOGY section. The outcomes of the study are reported in 
the RESULTS and CONCLUSIONS sections of this report. 

Dissemination 

Dissemination of the findings of this Assessment Study were 
achieved through a three step dissemination plan (see Appendix A) . 
Step one involved a minor revision of the CPSS package of materials 
to correct typographical errors and to include information about 
the Assessment Study. Step two required project staff to prepare 
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TABLE 1. Data Sources for the Assessient of CPSS. 



TIMING AND DATA SOURCE DURING ACADEMIC YEAR 



T 

OBJECTIVES 


At Beginning of 
the Year 


At Interim Points 
of the Year 


At the End of 
the Year 


1. Systematic career 
development program 
plans (experimental 
and control schools) 


1 Composite Status 
Reports generated 
by the school (CDPSR) 

• School records 

• Completion of on- 
site Verification 
Checklist by 
National Center 
project staff (VCL) 


f Verification by 
project staff via 
telephone, mail, 
or site visit 


• Plans/Reports 
generated by the 
schools (CDPSR) 

• School records 

• Completion of on- 
site Verification 
Checklist by 
National Center 
project staff (VCL) 


a. Knowledge of 
CPSS (experimental 
schools only) 


• Career Planning 
Support System 
Knowledge Test 
(steering committee 
only) 




• Career Planning 
Support System 
Knowledge Test 
(steering committee 
only) 


b. Attitudes toward 
systematic planning 
for career develop- 
ment programs (ex- 
perimental schools 
only) 


• Perceptions of 
Program Planning 
for Career Develop- 
ment Scale (steer- 
ing committee only) 




• Perceptions of 
Program Planning 
for Career Develop- 
ment Scale (steer- 
ing committee only) 


2. Career develop- 
ment program plans 
containing activi- 
ties having a high 
probability of im- 
proving students' 
career development 
skills (experimental 
and control schools) 


• Composite Status 
Reports generated 
by the schools (CDPSR 

• School records 

• Completion of on- 
site Verification 
Checklist by 
National Center 
project staff (VCL) 


• Verification by 
project staff via 
telephone, mail, 
or site visit 


• Plans/Reports gen- 
erated by the 
schools (CDPSR) 

• School records 

• Completion of on- 
site Verification 
Checklist by 
National Center 
project staff (VCL) 



and submit two journal articles about the study and to coordinate 
an informational seminar about CPSS at a national conference. 
Step three involved mailing a complimentary set of CPSS materials 
to each state and territory department of guidance. 
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METHODOLOGY 
Instrumentation 



Two data collection forms and three assessment instruments 
were developed to^ assess attainment of the project objectives. 
An overall listing of the forms and instruments, who completed 
them, the tx'me and sequence of their completion, and each objec- 
tive they address is presented in Figure 2. 

Data Collection Forms 

Two data collection forms were used to gather descriptive 
informatio-n from the participating schools both pre and posttest. 
These data sources constituted the inputs for executing the Career 
Development Program Rating Instrument. 

• The Career Development Program Status Report (CDPSR) . The 
CDPSR organized information about the school and community and 
about the ongoing and projected career development program planning 
efforts at participating schools. The construction of the CDPSR 
was based upon the kinds of descriptive information needed to 
determine systematic planning (as exemplified by CPSS) . The 
CDPSR (see Appendix B) was completed by school personnel both pre 
and posttest and consisted of four sections: 

I. Site Description - community, school district and school 



Figure 2. Forms and Instruments 



Data Collection Forms 



Pretest Posttest 
Completion Completion Intended 
Persons Completing Date Date Objectives 



• Career Development Program Status High School Staff (all 12/78 
Report (CDPSR) schools) 



6/79 



1 & 2 



• Verification Checklist (VCL) 



National Center Project 
Staff (all schools) 12/78 



6/79 



1 i 2 



Assessment Instruments 

• Career Development Program Rating Expert Panel of Raters 
Instrument 8/79 



8/79 



1 & 2 



• Career Planning Support System 
Knowledge Test 



• Perceptions of Program Planning 
for Career Development 



CPSS steering Coiranittee 
Menders (experimental 

schools only) 12/78 

CPSS steering Committee 

Members (experimental 12/78 
schools only) 



6/79 



6/79- 



la 



lb 



1 

In addition to the experimental school CPSS steering committee members, thirteen school 

o staff in three control schools completed these instruments at posttest only (see pp»50&54),, 
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II. Career Development Program Description - staff organiza- 
tion, state and district goals, and an overview of acti- 
vities and resources 

III* Present*'" Planning Efforts - personnel, assessment, and 
evaluation activities 

IV. Projected"^ Planning Efforts - perso.-^nel, assessment, and 
evaluation activities . 

By completing the CDPSR, participating schools documented 
the extent to which their existing career development program 
planning reflected the basic components of systematic career 
development program planning (staffing and personnel support, 
needs assessment, goal selection, development of behavioral ob- 
jectives, and career development activities). 

A pilot tryout of the form was completed with the assistance 
of two external consultants, both experienced career development 
practitioners. They were told that the form would be used by high 
school staff members to report information about the status of 
their career development programs and the programs' suitability to 
their communities. The consultants conducted their reviews of the 
CDPSR from the viewpoint of a director of guidance who might be 
asked to complete it for his or her high school. The consultants 
were instructed to assess the clarity of the structure, terminology, 
and phrasing of the report and the ease of retrieval of the requested 



In the pretest version, sections III and IV referred to the 1977- 
78 and 1978-79 school years, respectively. In the posttest 
version, these changed to 1978-79 and 1979-80, respectively. 
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information as well as to comment on other problems encountered 
in completing the report. 

Subsequently^ a revised version of the CDPSR was submitted 
for final review to two additional external specialists in instru- 
ment development and career guidance. The consultants' report 

r 

Stated that the form provided information related to the goals of 
a systematic career development program and that it appeared to 
have content validity. 

• The Verification Checklist (VCL) . This form provided a 
means by which a National Center project staff person could corro- 
borate ^ clarify^ and expand the information recorded on a school's 
Career Development Program Status Report. The design of the VCL 
was based on the essential components of systematic career develop- 
ment program planning as exemplified by CPSS (see Appendix C) . 

During the pre and posttest site visits, a National Center 
project staff member completed the VCL based on the information 
presented in the school's CDPSR. Subsequently, during a face-to- 
face meeting with school personnel who completed the CDPSR, the 
National Center staff member asked for any clarification, probed 
for any missing information, and attempted to attain the most 
complete and accurate information possible with respect to the 
school's career development program planning efforts. Finally 
both the staff member and school coordinator signed the VCL. 
In this manner the VCL provided evidence supportive of, and sup- 
plementary to, the self -reporting mechanism of the CDPSR. 



Assessment Instruments 

# Career Development Program Rating Instriiment (RI) , The 
Career Development Program Rating Instrument was used by a rating 
panel with expertise in career development to rate the adequacy, 
quality, and potential impact of a high school's career develop- 
ment program planning as reported through the descriptive data 
sources, the CDPSR and the VCL. 

The panelists were instructed to read all information pro- 
vided in a school's CDPSR and VCL before using the RI to rate th^ 
school. The RI consisted of 33 items and was divided into seven 
sections. Each section had directions that applied to the items 
in that section and listed specific-references in the data sources 
for each section or for individual items in a section* The data 
sources (CDPSR and VCL) were often accompanied by appendices that 
were attached to the CDPSR as part of the reference. 

The RI sections represented basic component areas of a 
systematic career development program planning effort. Concep- 
tually, sections I through V asked questions concerning specific 
facts describing the school's career development program. Section 
VI of the RI asks for the rater's estimate of the impact on 
student career development skills of future plans for the school's 
career development program. Section VII provided the raters an 
opportunity to express their judgments about the overall quality 
of career development program planning at a school relative to 
Objectives One and Two. Sections I through V, thus, were designed 
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to familiarize each rater with the facts, in preparation for the 
broad judgments requested in Sections VI and VII. 

Both discrete variables (dichotomous and categorical) and 
continuous variables comprised the RI items (see Appendix D) . 
Raters usually indicated their judgments on a four or five point 
Likert scale or by a Yes-No response unless specific instructions 
directed them otherwise. Each item was followed by a confidence 
rating which enabled the rater to indicate his/her confidence that 
the answer marked was correct. Therefore in addition to the 
reliability coefficients reported in the RESULTS section of this 
report, another estimate of reliability was obtained when panelists 
were asked to estimate their confidence in each rating they made. 
Raters were asked to place a check along a scale from zero to 
100 indicating their judgments regarding the likelihood that 
their answers were accurate. The format of the confidence rating 
is reproduced below. 

CONFIDENCE RATING: Please estimate the probability that your 
answer is correct by placing a check at the appropriate loca- 
tion on the scale. 

1 1 1 h 

0 10 20 30 
no 

confidence 

A prototype of the present RI was used by the rating panel 
in January, 1979 to rate the pretest data sources. However, due 
to unacceptably low reliability measures among raters, the original- 
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40 50 60 70 80 90 100% 

complete 
confidence 



rating instruments were revised with the help of an external test 
construction consultant and the pre and posttest data were rated 
in August, 1979, usiivg the revised RI . 

Principle concerns in the RI revision were: (1) to include 
only items that were clearly answerable given the CDPSR and VCL 
information, and (2) to eliminate items that did not allow control 
schools a fair opportunity to receive a high rating. .^Following 
examination and review by project staff, the RI was pilot tested 
by six National Center staff persons with expertise in instrument 
design, evaluation, and career development. The recommendations 
of these persons were incorporated in the final version of the RI . 

• Career Planning Support System Knowledge Test . A test of 
CPSS knowledge was constructed to assess Objective la (see p. 7 ) . 
A prototype multiple-choice test was developed based upon the 
contents of the Career Planning Support System Coordinator's 
Handbook , the principal reference for describing the concepts and 
procedures of CPSS. Since the test was intended to measure know- 
ledge of CPSS planning procedures, an effort was made by project 
staff to exclude items not related to the CPSS planning model. 
Any items of this nature detected later were deleted or not 
scored. 

The prototype test was reviewed first by a two-~person team 
of outside technical consultants followed by a single outside 
technical consultant. In response to the consultant recommenda- 
tions, action was taken to increase the coverage of CPSS planning 
steps, to eliminate items dealing with general planning procedures, 
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and to improve vaguely written items* On the advice of the 
consultants, the multiple-choice format was converted to a short 
answer (open-ended) one. 

A small-scale tryout of the test was conducted using two 
persons familiar with CPSS and two persons unfamiliar with it. 
After further revisicu, the test was reviewed for clarity of 
expression and ease of understanding by two school district 
persons present at the National Center for training as CPSS 
school coordinators. Their suggestions were incorporated in 
the final revision of the test. 

2 

The completed knowledge test consisted of 21 items with 
an average of three to four score points possible for each item 
(see Appendix E) . A total of 74 points was possible. 

The 74 possible points of the test were distributed across 
eight areas, with the number of items and points for each area 
determined, for the most part, by the approximate amount of 
time the steering committee was schedule (by the Coordinator ' s 
Handbook ) to meet regarding the topic. The points by areas were: 
career development units (17) , needs assessment (12) , resources 
(9), annual review (9), goals (8), objectives (7), overall planning 
coordination (7), student involvement (5). The results for the 
knowledge test are reported in the RESULTS section, p. 49. 



Originally, there were 27 items in the test. Six items (9, 11, 
13, 14, 21B, 18) were not used because of defects found 
after the test was printed. 
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• Perceptions of Program Planning for Career Development . 
A prototype attitude instrument was constructed with the help of 
an external consultant to assess the attitudes of CPSS steering 
committee members toward systematic planning for career develop- 
ment programs. This instrument was developed to meet Objective 
lb (see page 7 ) . Recommendations from a team of two external 
technical consultants, a single external technical consultant, 
and a National Center instrument design specialist were followed 
in revising and improving the instrument. 

The instrument consisted of 32 statements about program 
planning for career development (see Appendix F) . The statements 
reflected the planning concepts and procedures of CPSS as presented 
in the Career Planning Support System Coordinator's Handbook . 
Each subject had the opportunity to select one of five responses 
.(disagree strongly = -2, disagree = -1, neutral == 0, agree = +1, 
agree strongly = +2) for each statement. Scoring was accomplished 
by coding the five responses (+2 through -2) and summing across 
the 32 items. The possible range of scores was from -64 to +64. 
The results on the attitude instrument are reported in the RESULTS 
section, p. 5 3. 

Panel of Raters 

Selection of Panelists 

A major part of the Assessment Study involved the selection 
and recruitment of an external review panel of experts whose task 
would be to rate the adequacy, quality, and potential impact of 
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career development ' program planning and activities at experimental 
and control schools. A number of criteria for selection of review 
panel members were listed: proponent of comprehensive career guid- 
ance; national visibility; reputation within the profession; a 
mix of practitioners, administrators, and methodologists ; availa- 
bility; and experience in evaluation. Other considerations in- 
cluded minority representation, male to female ratio, and geographic 
distribution. National Center staff with familiarity in the area 
of career development were asked to suggest potential panel members 
meeting these criteria. A list of over 30 persons was compiled 
from which fifteen persons would be selected. 

In the initial telephone contacts, prospective panel members 
were asked if they would be willing to serve on a 15-member rating 
panel consisting of persons like themselvies with background and 
experience in the areas of career development. They were informed 
that the panel would be performing assessment activities on data 
received from high schools throughout the nation. 

The main reason for use of a panel of expert judges is related 
to the nature of the subject matter. Few people would doubt that 
efficient organization and planning comprise important aspects 
of high school career development programs. Yet the important 
features of efficient organization and planning remain uncodified 
in sufficient detail to permit completely objective measurement. 
In such instances, human judgments are essential. Hence, a, 
panel of individuals with the experience, training, and reputation 
was assembled to provide the most accurate judgments available. 
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Rating of Schools 

The rating panel convened at the National Center for Research 
in Vocational Education on January 24, 25, and 26, 1979 (see 
Appendix G for list of panel members) • The panel rated pretest 
data collected from the eighteen high schools. The most signifi- 
cant outcome of the first meeting was a subsequent revision of the 
Career Development Program Rating Instrument- 

A second meeting of the rating panel occurred August 20, 
21, and 22, 1979. Following brief introductions by project staff, 
the panelists were supplied sample copies of the revised Career 
Development Program Rating Instrument (RI) with accompanying 
definitions and directions. The RI and directions were reviewed 
with the panel so that all panelists were sure of their assignments 

The task given the panelists was to use a Career Development 
Program Rating Instrument to rate a school's career development 
program planning as reported through two data sources, the Career 
Development Program Status Report (CDPSR) and the Verification 
Checklist (VCL) . All panelists completed a rating instrument 
for each data set they reviewed. 

The panelists' (N=15) rating assignments were planned so 
that each school (N=18, pretest; N=18, posttest) was rated by 
three or four different and randomly-assigned raters. Assigning 
more than one rater to each school permitted numerical assessment 
of reliability of the ratings and yielded more accurate results 
than could be obtained from a single rating per school* Each 
panelist rated both the pre and posttest data from a school. 



Each panelist rated eight different data sets (one pretest, one 
posttest) from four different schools. 

Two hour time periods were alloted for the rating of a single 
data set. At the end of each rating period, each data set was 
collected and then redistributed to the next appropriate panelist. 

In order to eliminate rater bias, the panelists were not 
informed that a CPSS model provided the framework for this study. 
Neither were they advised of the pre-post, experimental-control 
design of the study. They were told that the data sources were 
reports taken at two different time periods from a set of high 
schools and that some of the schools in the two time periods may 
have been the same. However, all identifying information, e.g., 
state, city, school name, school address, dates, had been deleted 
prior to the ratings. 

In a final debriefing session, after all rating activities 
had been completed, the panelists were told that they had parti- 
cipated in an assessment study of the Career Planning Support 
System. They were given CPSS materials, a study abstract, and 
informed of all aspects of the study. The panelists indicated 
that they had neither surmised the nature of the study nor 
recognized that they had rated pre and posttest data from the 
same schools. 

A last task required of the panelists was the submission of 
a two-page paper of recommendations or comments to assess the 
rating procedure, rating instrument, implications for analysis, 
and the adequacy of the Status Report and Verification Checklist 
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as sources of information about a school career development 
program. These recommendations were used in preparing the JDRP 
report. 

E xperimental and Control School Selection 
The intent of the school selection process was to identify 
a sample of urban high schools^ ten experimental and ten control, 
to participate in the CPSS Assessment Study. Project staff ex- 
pected to select two volunteer high schools from each of ten 
urban school districts. Once identified, each high school was 
to be randomly assigned to either an experimental or control 
condition. The purpose in selecting one experimental high school 
and one control high school from the same district was to control 
for district and geographic variables and to minimize national 
travel. 

The expected school selection process was difficult to 
carry out. CPSS requires a staff commitment of one part-time 
coordinator and a working steering committee. Some school dis- 
tricts could not commit any staff to the project. Many school 
districts were experiencing budget and subsequent staff cutbacks, 
union/school board problems, and desegregation difficulties. Other 
school district officials cited the passage of California's 
Proposition 13 as a detriment to participation in a national 
project of the scope of the CPSS Assessment Study during 1978-79 • 
These factors caused project staff to make compromises in obtain- 
int the desired number of schools within optimal research conditions. 
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School Selection 

Project staff began the school selection process on July 21, 
1978, by inviting superintendents of school districts in thirty- 
five of the fifty largest cities in the United States to recommend 
high schools in their districts to participate in the national 
assessment of CPSS (see Appendix H) . School district superinten- 
dents were encouraged to nominate interested high schools. Each 
interested school would complete a School Information Checklist 
(SICL) which reported information about the school relative to 
the number of faculty/staff, size of the student population, 
distribution of ACT/SAT scores, drop-out rates, estimated average 
family income of student population, racial/ethnic mix of student 
population and other information (see Appendix I) . Project staff 
ultimately used the SICL to comparatively match schools within 
districts. The fifteen city school districts which did not re- 
ceive the initial mailing were known to be experiencing some of 
the difficulties mentioned in the previous section. By selecting 
high schools from districts in the largest cities in the United 
States, it is expected that the results of the study may generalize 
to the widest population of American high schools. 

The initial mailing produced few candidate schools by mid- 
September. To enlarge the sampling frame and to broaden geographic 
distribution, additional school districts were invited to parti- 
cipate in the study. These districts were located in the standard 
metropolitan areas (SMAs) surrounding some of the large cities. 
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By mid-October eighteen high schools representing eight 
school districts agreed to participate in the study. The final 
two schools, each representing one of two adjacent school districts, 
were xocated in a rural setting and thus expanded the domain of 
representation. Table 2 shows the number of experimental and 
control schools, the school districts, cities, and states that 
participated in the CPSS Assessment Study. 

Experimental and Control Assignment 

Of the eighteen high schools, four schools were randomly 
assigned to experimental and control conditions. Most local 
school district administrators insisted that the determination 
of experimental auC control schools be decided at the local district 
level, rather than through random assignment by the National Center; 
therefore, fourteen schools were not randomly assigned to condi- 
tions (see Table 2). Some of the local considerations for assign- 
ment to conditions were: 1) staffing patterns in the experimental 
schools allowed for easy assignment of a CPSS coordinator; 2) the 
experimental school was ready to start with CPSS, the control 
school wanted to wait; 3) a school dropped out at the last minute 
and the remaining school was assigned experimental, until a control 
school could be found. Nevertheless, schools within districts 
appeared closely matched on the characteristics reported on the 
School Information Checklist (see Table 3) • Table 3 lists the 
descriptive information obtained from the SICL about each of the 
schools involved in the study. 



TABLE 2 

Experimental and Control Schools 



District 


City and State 


5 of FxDprimpTitril 
Hiqh Schools 


1 of Control 
■Hicjh Schools 


Assignment 
to Conditions 


Chicago Public Schools^ 


Chicago, IL 


2 


2 


Random 


1 

Baltimore City Public Schools 


Baltimore, MD 


2 


1 


Nonrandom 


Dade County Public Schools^ 


Miami, FL 


2 


1 


Nonrandom 


Jefferson County Public Schools 


Louisville, KY 


1 


1 


Nonrandom 


Memphis City Schools 


Memphis, TN 




1 


Nonrandom 


Jefferson County Public Schools 


Lakewood, CO 




1 


Nonrandom 


2 

Globe Public Schools 


Globe, AZ 






Nonrandom 


2 

Miami Public Schools 


Miami, AZ 




1 


Nonrandom 



'To insure a sufficient number of experimental and control schools, project staff accepted all of 
the schools reconiended by interested school districts. 

'Globe and Miami are adjacent districts with one high school in each district. For the purposes 
of the assessment study, Globe and Miami were matched, Globe is the experimental high school 
and Miami is the control high school. 
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TABLE 3 



Characteristics of Experimental ajd Control 
Schools in Each District 



2 ACT Student 

School TreatT Student Teacher s/Coun- Racial/Ethnic Composite or Est. Average Grade Drop-Out 

Code ment Pop. selors/Administrators (Percentages) SAT Family Income Levels Rate Stfite 

Tchrs/Counselors/Adrain W / B / SP / Asian / Nat. Am (in thousands) 



01 


X 


1029 


48 


2 


2 


54 


- 


21 


- 


25 17.5 


10-15 


9-12 


121 _ 


A7;' 


02 


c 


617 


35 


1 


2 


58 




40 




1.4 19.4 


10-15 


9-12 


8.9% 




03 


X 


1278 


72 


5 


3 




100 


- 


- 


350/300 


15-20 


10-12 


51 




04 


Y 

A 


2006 


101 


7 


4 


10 


90 






330/330 


5-10 


9-12 


n 


m 


05 


c 


3000 


120 


7 


5 


30 


70 






300/337 


5-10 


10-12 


m 




06 


Y 




110 


U 


5 




100 

J* V V 






12 


5-10 


9-12 


2M 




09 


c 


1853 


80 


4 


3 




100 






10 


5-10 


9-12 


16.27. 


IL 


07 


X 


3054 


159 


8 


5 




100 


- 


- 


12 


10-15 


9-12 


It 


08 


c 


2735 


139 


7 


4 




100 






14 


10-15 


9-12 


n 




10 


X 


1600 


92 


5 


5 


83 




16 


1 


19.3 


10-15 


10-12 


57, 


CO 


11 


c 


2258 


101 


6 


5 


86 




13 


1 


19.3 


15-20 


9-12 


10.9'^ 


12 


X 


1950 


88 


4 


4 


70 


30 






16.5 


5-10 


9-12 


n 


KY 


13 


c 


1406 


80 


4 


3 


77 


23 






18.2 


10-15 


9-12 


bi 


14 


X 


893 


63 


3 


4 


72 


28 






16.7 


15-20 


9-12 




TN 


15 


c 


1399 


55 


3 


3 


85 


15 






18.4 


15-20 


9-12 


5Z 


16 


X 


2300 


109 


7 


4 


30 


40 


30 




N/A 


10-15 


10-12 


147. 




17 


X 


3000 


138 


6 


5 


62 


24 


12 


2 


^40 (mean) 


10-15 


9-12 


11% 


FL 


18 


c 


??74 


116 


5 


4 


70 


17 


11 


2 


507 (mean) 


15-20 


10-12 


m 





Districts are shown by horizontal broken lines separting groups of schools 

1 

Ethnic abbreviations: W = White, B = Black, SP = Spanish Surname, Nat. Am = Native American 
■ Schools 01 and 02 are the two rural, Arizona high schools located in separate, adjaoeni- school districts 
4 

X = experimental, Z = control 
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Whenever random assignment to treatment groups cannot be 
achieved, observed differences between treatment groups in theory 
can be due to nontreatment variables. The standard methodology 
for handling objections of this sort is to introduce some type 
of statistical control for a small group of variables that are 
likely candidates to account for observed differences between 
treatment groups. In the present case, the sample size is small 
enough to render such procedures of dubious value. One may 
observe, however, bivariate relationships between selected "control" 
variables and the treatment variable. 

In the present study the treatment variable is defined by 
the two categories — used CPSS and did not use CPSS. Averages on 
the following variables were compared statistically for users and 
nonusers of CPSS: student population, faculty/student ratio, 
academic test scores, drop-out rate, percentage of the student 
population who were minority group members, and a rough estimate 
of family income of the student body. As shown in Table 4, in 
none of these six tests were statistically significant differences 
observed. Hence, it is concluded that the differences between users 
and nonusers of CPSS reported in the RESULTS section of this report 
are not likely due to any of these six characteristics of schools. 

Experimental and Control School Implementation 

C ontracts 

Once schools were selected and assigned to experimental 
and control conditions, project staff initiated a contractural 



TABLE 4 

Key Characteristics of Test Sites 



Average of 
Characteristic 


Control 
Schools 


Experimental 
bcnoois 




~vali 


Student population 


ly lb 


ly 4 J 




.0/4 


Ratio of faculty and 
staff to student 
population 


19.49 


17.64 


1 


. 540 


ACT/SAT scores* 


15.67 


16. 84 




.748 


Estimated family income 


$12000 


$13125 




.607 


Drop-out rate 


7.4% 


10.0% 


1 


. 375 


Percent white 


38% 


51% 




. 814 



NOTE: Table entries are averages over the control or ex- 
perimental schools, as labeled. Experimental school 
refers to a school that used CPSS during the study, 
and control school refers to a school that did not 
use CPSS. 



*Five schools made SAT scores available, and the remaining 
13 submitted ACT averages. The five SAT scores were 
converted to the metric of ACT by dividing them by 
the ratio of the average over schools SAT to the 
average ACT. 



31 




relationship with each of the school districts involved through 
purchase of services agreements. These were used to help the 
participating schools defray some of the costs related to the study 
$5,000.00 was set aside for each experimental school to reimburse 
the school district for staff time, and $500.00 was set aside for 
each control school for reimbursement of staff time, related to data 
collection. 

Each school was required to provide a part-time coordinator. 
In the experimental schools, the coordinator was responsible for 
the preparation and completion of data collection forms and served 
as coordinator of CPSS in the school. In the control schools, 
the coordinator was responsible for the completion of the necessary 
data collection forms and served as the on-site contact for the 
study. A copy of the work statement which accompanied the purchase 
of services agreement is appended (see Appendix J). 

Pretest Data Collection 

With purchase of services agreements initiated for each 
experimental and control school, National Center project staff 
began the on-site pretest data collection. As mentioned on p. 13 , 
each school completed the Career Development Program Status 
Report (CDPSR) as a pretest report of the school's career develop- 
ment program. Project staff, using the Verification Checklist 
(VCL) , reviewed the CDPSR on-site to insure that the CDPSR was 
a complete self-report of the career development program in the 
school. The purpose of the VCL was to highlight the strongest 



aspects of the school's procjram. The VCL was signed by both the 
school representative and the project staff member. 

These pretest visits were conducted between mid-October and 
mid~November in sixteen of the eighteen schools. Two control 
schools were not able to complete the pretest data reporting 
until mid-December (see Figure 1, p. 7). 

At pretest, nine of the eighteen schools received on-site 
visits and completed data collection activities prior to assign- 
ment to experimental or control conditions. Of these nine, four 
were randomly assigned and five were not. The other nine schools 
completed pretest data collection activities after assignment to 
experimental or control conditions. 

The pretest data collected on the CDPSR and VCL provided 
the information that the rating panel used to rate the eighteen 
schools prior to the CPSS treatment. 

In the experimental schools, additional pretesting was per- 
formed. Each CPSS coordinator gave the steering committee members 
a timed pretest (45 minutes) relative to the members' knowledge 
about CPSS procedures. The Career Planning Support System Know - 
ledge Test , and their attitudes toward systematic planning for 
career development programs. Perceptions of Program Planning for 
Career Development . These tests were not given to control school 
staff (see pp. 19,21.. for descriptions of the instruments). 

Training 

The ten CPSS coordinators from the ten experimental schools 
were brought to the National Center in November, after the pretest 



visits, to receive training in CPSS procedures. The three-day 
training involved an introduction to the rationale and basis for 
the CPSS model, an in-depth review of each step in the CPSS 
process and "hands-on" practice with the CPSS materials. Once 
the CPSS coordinators returned to their schools, after training, 
they began to install CPSS. 

Site Monitoring 

The experimental and control schools were monitored through-- 
out the year (November, 1978 - June, 1979) up to and including 
posttesting. In the experimental schools, CPSS coordinators 
completed project logs twice a month, describing the progress of 
CPSS in, the school. In addition. National Center project staff 
telephoned each school at least once per month and conducted 
an interim on-site visit during February. Project staff completed 
an on-site monitoring log for each school visited. The on-site 
monitoring log described the nature and scope of the visit. 

For the control schools, site monitoring involved telephone 
calls on the average of one per month and an on-site visit in 
February. An on-site monitoring log was completed for each 
control school. The purpose of the on-site visit to the control 
school was to review at the school the on-going activities related 
to the career development program in the school. 

Technical assistance to experimental or control schools was 
minimal. Although the experimental schools received three days 
of training in CPSS, proj^t staff did very little to assist 
experimental schools in the installation of CPSS. 
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PosttGSt: Data Colloction 

The final phase of experimental and control ^5chool activity 
was the posttest data collection. Each school completed a post- 
test version of the CDPSR. This form was exactly the same as 
the pretest version except that dates were changed to reflect the 
different reporting period. The information required on the form 
was the same as the pretest. National Center staff conducted 
posttest site visits to collect the CDPSR and reviewed it on-site 
to insure that it was completed fully. The VCL was used to 
insure that all possible information was reported in the CDPSR. 
As during the pretest, the VCL was signed by both project staff 
and the school coordinator. 

In the experimental schools, additional posttesting occurred. 
National Center staff administered to steering committee members 
the posttest versions of the knowledge test of CPSS procedures 
and the attitude test. Also, post-project interviews were con- 
ducted with steering committee members to gather information about 
the use of CPSS in the school. These interviews gave steering 
committee members a chance to describe the strengths and weaknesses 
of CPSS and to report their experiences with CPSS. In addition, 
two career development units prepared in experimental schools as 
a direct result of CPSS were comparatively tested in each of the 
experimental schools. This information was collected, in part, 
to address Objective Two (see p. 7 ) . Each experimental school dev- 
oped two career development units based on th r students' needs. 

o 
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Each unit involved instruction for students in some career skill, 
i.e., career exploration, interviewing. The teacher-developed 
test of achievement for each career development unit was given to 
the class that had been taught the unit and also to a comparable 
class which had not been taught the unit. The result of these 
comparisons is reported in the RESULTS section, p. 48. 

In the control schools, posttesting involved completion of 
the CDPSR and cooperative completion of the VCL. Each of the 
control schools was given a set of CPSS materials and an orien- 
tation to CPSS once the posttest data collection was completed. 
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RESULTS 
Rating Instrument 

Major Hypotheses 

There were two major hypotheses tested by use of the Career 
Development Program Rating Instrument- The first major hypothesis 
maintains that school staffs using CPSS will produce greater 
change toward a systematic plan for career guidance than will 
school staffs not using CPSS. This corresponds directly with 
Objective One, p. 7 . The second major hypothesis states that 
school staffs using CPSS will develop higher quality activities 
to improve student career development skills than will school 
staffs not using CPSS. This hypothesis related to Objective 
Two, p. 7. The numbers used in the analyses for these hypotheses 
are the averages over raters of scores for a given item on the 
rating instrument. 

The key items on the rating instrument related to the first 
hypothesis were those taken from section VII of the rating 
instrument, items 2 8 through 33 • These were the rating items 
that required the raters to make judgments about key components 
of a systematic plan for career guidance: staff organization, 
needs assessment, goal setting, preparation of behavioral objec- 
tives ,n and design of effective career development activities. 
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The main rating items for analysis of the first hypothesis are 



reproduced below. 



28. Estimate the extent to which the school staff was or- 
ganized to plan systematically a comprehensive career 
development program by evidence of clearly designated 
.. leadership; administrative cooperation; and permanent, 
active groups and committees. 

( ) ( ) ( ) 



{ ) 
(0) 

Limited Extent 



(1) 



(2) 



( ) 

(3) (4) 

Great Extent 



29. Estimate the extent to which a student career develop- 
ment needs assessment was conducted, tabulated, properly 
interpreted, and the data utilized for planning the 
career development program. 

( ) ( ) ( ) 



( ) 
(0) 

Limited Extent 



(1) 



(2) 



( ) 

(3) (4) 

Great Extent 



30. Estimate the extent to which a comprehensive set of 

ordered career development goals reflecting assessed 

student career development needs were developed and 

used in planning, implementation and evaluation of 
the program. 

( ) ( ) ( ) ( ) ( ) 

(0) (1) (2) (3) (4) 



Limited Extent 



Great Extent 



31. 



32. 



Estimate the extent to which a set of behavioral ob- 
jectives was developed reflecting specific goals and 
containing a clear statement of the intended audience, 
behavior, situation and standard of mastery. 



( ) ( ) 

(0) (1) 

Limited Extent 



( ) 
(2) 



( ) ( ) 

(3) (4) 

Great Extent 



Estimate the extent to which career development activi- 
ties were developed that reflect student needs, goals, 
and associated objectives, and that indicate methods, 
target student group, and outcome measures by referring 
to the two attached career development activities 
(yellow) . 



( ) ( ) 

(0) (1) 

Limited Extent 



( ) 
(2) 



( ) ( ) 

(3) (4) 

Great Extent 
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33. Based on the available information (including all yellow 
and pink career development activities) , rate the 
overall quality of the school's career development 
program. 

( ) ( ) ( ) ( ) ( ) 

(0) (1) (2) (3) (4) 

Very Low Quality Very High Quality 

The rating items used to support the second hypothesis were 

items 19, 27 and 32. These were the items that required the 

expert raters to make judgments about the quality of career 

development activities. The items for analysis of the second 

hypothesis are reproduced below. 

19."^ What is your best estimate of the chance that each of 
the career development activities will improve student 
career development skills, irrespective of whether the 
career development skill is listed as an objective 
of the activity. 

Activity A: ( ) ( ) ( ) ( ). 

Activity B: ( ) ( ) ( ) ( ) 

Good Some Little Very Little 

Chance Chance Chance Chance 

(3) (2) (1) (0) 

27. If the plans reported on SR pp. 16-20 are carried out, 
what, in your experience, is the chance that student 
career development skills will be improved? 

{ ) ( ) ( ) . ( ) 

Good Some Little Very Little 

Chance Chance Chance Chance 

(3) (2) (1) (0) 



Item 19 has answer choices A and B. If a school reported two 
career activities both A and B were used. However, if a 
school reported one career activity, A was used. If none 
were reported no activity was rated; therefore, more "cases" 
are reported for 19A {N=33)than for 19B (N=27) . 
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32. Estimate the extent to which career development activi- ' 
ties were developed that reflect student needs, goals, 
and associated objectives, and that indicate methods, 
target student group and outcome measure by referring 
to the two attached career development activities 
(yellow) . 

( ) ( ) ( ) ( ) ( ) 

(0) (1) (2) (3) (4) 

Limited Extent Great Extent 

Data Analysis Methods 

Two statistical methods were used to assess the two hypotheses,* 
The most straightforward method is a two-factor analysis of 
variance (ANOVA) in which one factor i^^ time (pretest and post- 
test) and the other factor is experimental condition (used CPSS 
and did not use CPSS) . The research hypothesis for the ANOVA 
is that a statistical interaction will be observed of the follow- 
ing form: average change on rating scores toward a systematic 
planning process or better career activities from pretest to 
posttest for experimental school staffs will exceed that for 
control school staffs. Thus, the F test for interaction in the 
two-factor ANOVA will be the one on which to focus. Since the 
form of the interaction is predicted in advance, it is equally 
important to observe the means for each of the four cells 
in the. design. The pattern of these means should conform to the 
expectation that experimental schools exhibit greater gain than 
control schools. 

The second statistical method for assesssing the two hypotheseo 
is analysis of covariance (ANCOVA) . The dependent variables for 
the ANCOVA are posttest rating scores describing the planning 
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process or career activities of each school at the end of the 
experiment. The ANCOVAs contain one categorical factor — experi- 
mental condition defined by the use or nonuse of CPSS — and one 
covariate defined as the pretest rating score corresponding 
to the posttest dependent variable. Conceptually, the ANCOVAs 
describe differences in posttest rating scores between school 
staffs using CPSS and school staffs not using CPSS, under statis- 
tical control for the pretest rating score . 

Although it does not appear to be widely recognized, the 
ANCOVA model can be viewed as a model of change, just as is the 
two-factor ANOVA described above. Conceptually, the ANCOVA can 
be viewed as expressing the following hypothesis: Change over 
the period of the experiment is greater for school staffs using 
CPSS than for school staffs not using CPSS, when statistical con- 
trols for the effect of the starting point (pretest rating scores) 
are applied- 

« 

In addition to the analysis of variance and covariance, re-- 
liability estimates of the rating scores are reported. The unit 
of analysis for all statistical results is defined by schools. 
A score describing each school on each variable is calculated 
by forming the average over the three or four raters who rated 
each school. The agreement among raters for a given school 
indicates the reliability of the scores, and, conversely, 
disagreement among raters indicates unreliability. The discre- 
pancies among raters of a given school can be compared to differ- 
ences in average ratings across schools. This basic idea forms 
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the conceptual basis for calculating reliability coefficients 
based on an analysis of variance model (see Winer, 1971: 283 f f ) . 
The idea is to compare a mean-square within schools to the mean- 
square between schools. Since the object of the design is to 
minimize pretest differences among* schools, these calculations 
are based on posttest rating scores only. The calculations omit 
consideration of "anchor points" (Winer, 1971: 289 ff ) , thus 
yielding somewhat conservative estimates of reliability (see 
Figures 3 and 4) . The formula used approximates an unbiased es- 
timate of reliability, assuming no anchor point differences among 
raters (unlike correlational methods such as split half or coef- 
ficient alpha) . 

Findings 

The major results obtained for Hypothesis I are summarized 
in Figure 3. Each of the top five panels of the figure siimmarize 
the results for one element used to define a systematic plan for 
career guidance. The sixth panel suimnarizes judgments of the 
overall quality. The panels of the figure are numbered and 
labeled to correspond to the key rating items reproduced on pp. 38 & 39, 

The lefthand graphs plot subgroup means associated with the 
two-factor ANOVA. The line for the schools using CPSS is labeled 
E, for experimental group, and the line for schools not using 
CPSS is labeled C, for control group. The horizontal axis 
represents time, t = 1 Cor pretest, and t = 2 for posttest. 
The vertical axis represents the five-point scale (0 - 4) associ- 
ated with each item. The slope of the lines thus give the average 
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ANOVA 



ANCOVA 



Organization 




4-r 

3-- 

2- 

1- 

0"^ 



P< .001 



reliability = .852 



average 

confidence 

rating 



= 87.0% 





reliability = .922 



average 

confidence 

rating 



= 91.9% 



Goats 





reliability = .932 



average 

confidence 

rating 



= 91.5% 



I. Objectives 




P< .001 




reliability = .900 



average 

confidence 

rating 



= 90.7% 



2. Activities 




^ t 



P< .001 



reliability = .850 



average 

confidence 

rating 



= 88.0% 



3. Overall 
Quality 



4 1 
3- 




4-r 

3- 


2- 


. P <^^^^00^ E 


2- 


1 - 




1- 


0 


' 1 2 ' 


0 



^ reliability = .829 

P < .001 

average 
confidence 

rating = .87.7% 
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FIGURE 3 ANOVAs, ANCOVAs, reliabilities, and average 
confidence ratings for Hypothesis One. 
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change from pretest to posttest for schools using CPSS (E) and 
for schools not using CPSS (C) . Lack of parallelity of these two 
lines reflects the statistical interaction postulated for the 
AN OVA. 

In every case the slope is substantially steeper for experi- 
mental schools than for control schools. As indicated on the 
graph by the notation p < -001, the chance that the observed dif- 
ferences in slope are due to sampling error is less than one in 
1000. (These probabilities are reported for the interaction term 
in the ANOVA) . 

The righthand coluiTin of graphs displays plots of mean dif- 
ferences in posttest scores between experimental schools (E) 
and control schools (C) , as adjusted statistically by the analysis 
of covariance for pretest scores on the dependent variable. 
Alternatively, as noted above, these graphs may be interpreted 
as differences in change from pretest to posttest, adjusted for 
differences in starting point. The vertical axis of these graphs 
matches those for the ANOVAs. The horizontal axis does not re- 
flect a continuous slope. Rather, the lefthand point (labeled C) 
corresponds to the control group, and the righthand point corre-- 
spends to the experimental group (labeled E) . This positioning 
of E and C is arbitrary, but was selected so that a positive 
slope indicates support for the first hypothesis: that experi- 
mental schools show larger gains, when adjusted for starting point 
than do control schools. All six graphs do show a substantial 
positive slope , thereby lending support to the hypothesis. As 
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with the ANOVAs, all statistical tests are highly significant, 
with probabilities approaching zero. (Reported probabilities 
are for the main effect of the experimental variable^ after ad- 
justment for the covariate) . 

The reliability coefficient associated with each dependent 
measure is high, ranging from .829 to .932, and averaging .881. 
Also, the average confidence ratings of panelists is reasonably 
high, thus reinforcing the reliability calculations. In spite 
of the need for approximate judgments, therefore, it is concluded 
that available evidence is consistent with the view that the 
measurements are accurate to within tolerable limits. 

The results secured for Hypothesis II are summarized in 
Figure 4. Each of the three panels display ratings for the three 
key xtems related to quality career development activities. The 
panels of the figure are numbered and labeled to correspond to 
the rating items reproduced on pp. 39 and 40. 

Similar to Figure 3 , the lefthand graphs plot subgroup means 
associated with the two-factor ANOVA. The line" for the schools 
using CPSS is labeled E, and the line for schools not using CPSS 
is labeled C. The horizontal axis represents time, t = 1 for 
pretest, and t = 2 for posttest. The vertical axis represents 
the four point scale (0 - 3) for items 19A, 19B and 27 and a 
five point scale (0-4) for item 32, The slope of the lines 
thus gives the average change in the quality of career develop- 
ment activities from pretest to posttest for schools using CPSS 
(E) and for schools not using CPSS (C) . Lack of parallelity for 
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ANOVA 



ANCOVA 



19. 

Activity A 




1— t 



Activity B 




27. 
Skills 



3 
2 
1 

0 




32. 

Activities 




3 - 
2 - 
1 
0 



p< .008* Reliability = .751 



Average 

Confidence 

Rating 



= .846 



3 f 
2 
1 
0 



p < .095 Reliability = .645 



Average «■ 
Confidence 
Rating 



= .823 



4 
3 
2 
1 
0 



p < .003^ 



Reliability = .754 



C E 
p < .001 



Average 

Confidence 

Rating 



= .867 



Reliability = .850 

Average 
Confidence 
Rating = .880- 



FIGURE 4. ANOVAs, ANCOVAs, reliabilities, and average 
confidence ratings for Hypothesis Two. 
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those tv/o lines reflects the statistical interaction, postulated 
for the ANOVA. 

In items 19A, 19B, and 21, the differences between experimental 
and control schools were not statistically significant. However, 
in each instance, a positive slope is observed, in favor of the 
experimental schools, thus lending some support to the research 
hypothesis. The graph of item 32 indicates a high level of 
significance. The probability that the observed differences 
were due to chance is less than one in 1000. The positive slope 
on this item indicates support for Hypothesis II: that experi- 
mental schools may produce higher quality activities than do 
control schools. 

The righthand column of graphs are also similar to those 
in Figure 3. This column displays plots of mean differences in 
posttest scores between experimental (E) schools and control (C) 
schools, as adjusted statistically by the analysis of covariance 
(ANCOVA) for pretest scores on the dependent variable. The ver~ 
tical axis of these graphs matphes those for the ANOVA. The hori- 
zontal axis does not reflect a continuous scale; rather, the left- 
hand point (C) corresponds to the control group and the righthand 
point (E) corresponds to the experimental group. Item 19B 
displays nonsignificant findings. However, as pointed out in 
the footnote on p. 39, item 19B is based on a smaller number of 
cases (N=27), than item 19A (N=33) . It may be that, due to 
this difference in N, larger gains would have to develop to show 
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Significant results. Items 19A, 27 and 32 all show significance 
at the .01 level. These findings support Hypothesis II. 

Reliability coefficients for items 19A, 19B, 27 and 32 
are also reported on Figure 4. These reliability measurements 
are within tolerable limits, .75 and above, except for item 19B, 
Again, this may be due to smaller N for the item. Average confi- 
dence ratings appear to be reasonably high and reinforce the 
reliability calculations. 

Additional information, reported below, was collected from 
experimental schools that reinforces the positive trend reported 
in the analyses for Hypothesis II. As reported on p. 36, Posttest 
Data Collection , experimental schools conducted comparison tests 
of two career development units (CDUs) developed as a result of 
the use of CPSS. In every case, the* achievement of the students 
who had been taught the CPSS-derived unit was much greater than 
for those students who had not been taught the unit. Each ex- 
perimental school conducted these comparisons in a different way, 
with different content, and on different populations; therefore, 
no effort has been made to report "findings." It is simply noted 
that differences were observed in tests of comparison between 
students who had been taught certain career development units 
based on CPSS and those who had not been taught the units. 
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Knowledge and Attitude Tests 

Objectives la and lb 

The knowledge and attitude instruments were designed to 
assess Objectives la and lb and, ultimately, to produce evidence 
^npnortive of Hypothesis I. It is assumed that an increase 
in CPSS steering committee members' knowledge during one year 
(as reported by scores on the Career Planning Support System 
Knowledge Test) and no change or a positive change in attitudes 
by these members toward systematic planning for career development 
(as reported by scores obtained on the Perceptions of Program 
Planning for Career Development scale) provide additional evidence 
to support the expectation that experimental schools display 
greater change toward the demonstration of systematic planning 
for career guidance programs than do control schools. 

Analysis Of Knowledge Test Scores 

The "Career Planning Support System Knowledge Test" was 
taken by 49 subjects at the pretest and 47 at the posttest. All 
were members of the steering committees (except for the CPSS 
school coordinators) at the ten experimental 3:::hool sites. Scores 
were also obtained from three control schools at posttest for 
comparison purposes. The knowledge test has acceptable reliability: 
correlation of split halves gave an estimated reliability coeffi- 
cient of .75, corrected for length. The small number of test 
items (21) and the large number of unanswered items (the items 
were open-ended) probably contributed to the reliability level. 
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Three project staff members scored the test. All three 
scored the five items that were viewed as most difficult (i.e.^ 
required more judgment) to score. Each of the remaining items 
was assigned to individual raters^ i.e.^ one rater- scored given 
items on all response sheets. For the five items scored by all 
three raters^ the median score was used. Intercorrelations 
between pairs of the raters on the five difficult items ranged 
from .78 to .84. 

Descriptive statistics are presented in Table 5 and Figure 5. 
As shown in Figure 5, the difference between pre and posttest 
scores is significant not only in the statistical sense ^ 
^(matched pairs) = 8. 23 , df = 9 , p < -001, but also in an "edu- 
cational" sense by showing that more than one standard deviation 
difference exists between means of pre and posttest measures 
(i.e.^ gains averaged over six points on the test from pre to 
posttest) . 

These findings support Objective la, which states that 
steering committee members' knowledge of CPSS will increase. 

Members of the school staff at three control schools also 
took the knowledge test at posttest time only. The mean know- 
ledge score for control schools was 8.7; the standard deviation, 
5.28 (see Table 6). These control school scores are quite 
similar to the pretest experimental school scores. 
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Table 5 



Mean Scores on CPSS Knowledge Test of 
Steering Committee Members in Experimental Schools 



Mean Score (over 
School people within school) 

Code Pretest Posttest 



01 


11. 15 


15.09 


03 


11.92 


20.91 


04 


16. 46 


19.72 


06 


10.57 


19.67 


07 . 


9. 37 


16.71 


10 


13.56 


21.31 


12 


8.28 


13. 65 


14 


8.93 


21.03 


16 


10.11 


21. 08 


17 


5. 76 


14.46 



Pretest 

mean = 10.26 (over schools) 

range of means = 5.76 to 

16.46 

range of raw scores = . 5 

to 2 8.7 

n = 49 
SD = 5. 53 



Posttest 

Mean = 18.13 (over schools*) 

range of means = 13.65 to 

21.31 

range of raw scores = 2.5 

to 34. 5 . 

n = 47 
SD = 6.J.6 
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Table 6 

Control Schools Knowledge Scores 



School Code Mean 

02 
05 
08 

Grand mean = 8.7 

Range of raw scores = 2 to 20 

n = 13 

SD = 5.28 

j\nalysis of the Attitude Instriament Scores 

All experimental school steering committee members (except 
the CPSS coordinators) completed the attitude scale both pre 
and posttest. Each of the 32 items on the "Perceptions of Program 
Planning for Career Development" allowed five possible degrees 
of agreement. Responses to these items were coded -2 through 
+2 and were summed to give a total score. The possible range 
of scores was from -64 to +64. Forty-nine response sheets were 
scored and double-checked for accuracy for the pretest; 47 for 
the posttest* This attitude scale also has acceptable reliability: 
the coefficient of reliability between odd items and even items, 
corrected for length, was .90. 

As shown in Figure 6 , the attitude scale produced a broad 
range of scores, distributed in a fairly normal manner on both 
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the pre and posttest. Table 7 shows the mean scores and other 
descriptive statistics for each school on both pretest and 
posttest. 

A t-test for matched pairs showed a significant difference 
between pre and posttest attitude scores (t = 2.821, df = 9, 
p .05), with the posttest significantly higher. The "educational 
sicjiiif icance" of this difference is less marked than the knowladj^ 
test; the difference between pre and posttest means is only 
•4 of one standard deviation (i.e., the gain averaged about 
four points on the attitude test from pretest to posttest) . 

These findings support Objective lb, which predicted an 
increase or no change in positive attitude scores after the 
CPSS program. 

Three control schools also took the attitude scale at 
posttest time. The mean score for these was 37.2; the standard 
deviation 9.5 (see Table 8). As with the knowledge tests, 
these scores are very similar to the pretest experimental school 
scores (Table 8). 
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Table 7 



Mean Scores on CPSS Attitude Scale of 
Steering Committee Members in Experimental Schools 



School 
Code 



Mean Score 
Pretest Posttest 



01 


43.5 


47. 8 


03 


42.0 


49.0 


OA 


51. 3 


44.8 


06 


33.2 


43.3 


07 


29. 4 


35.8 


10 


39.7 


46. 7 


12 


28.6 


35.7 


14 


42.2 


41. 8 


16 


38.0 


39. 3 


17 


38. 1 


46.0 



Pretest 



Posttest 



mean = 38.6 



mean = 42.8 



range of means = 2 8.6 to 

51.3 



range of means = 35.7 to 

49.0 



range of raw scores = 15 

to 61 



range of raw scores = 11 to 

59 



n = 49 



n = 47 



standard deviation of 
raw scores = 10.9 



standard deviation of raw 
scores = 11. 1 
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Table 8 

Attitude Scores for Control Schools 



School Code Mean 

02 36.3 

05 32.8 

08 42.0 



Grand mean = 37.15 

Range of raw scores = 26 to 60 

n = 13 

SD = 9.47 
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CONCLUSIONS 

The Career Planning Support System Assessment Study was 
designed to test the effectiveness of CPSS and was guided by two 
objectives stated as hypotheses. The first hypothesis stated 
that school staffs using CPSS for one year would develop a sys- 
tematic plan for career guidance. A subset of related objectives 
addressed experimental school staffs' knowledge of CPSS and their 
attitudes toward systematic planning. It was assumed that in- 
creased knowledge and favorable attitudes would reinforce school 
staffs' use of a systematic plan for career guidance. 

The second hypothesis stated that school staffs involved in 
CPSS for one year would develop a plan that contained activities 
having a high probability of improving student career development 
skills. 

The findings reported for Objectives One, la, and lb support 
the associated research hypotheses with p < .001, p < .001, and 
p < *05 levels of significance, respectively. These results suggest 
that during one year school staffs with school characteristics 
similar to the schools and staffs involved in the Assessment 
Study can successfully use the CPSS materials to generate a sys- 
tematic plan for career guidance as defined by the successful 
completion of key planning steps: 
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1. organization of staff 

2. needs assessment 

3. goal setting 

4. preparation of student behavioral objectives 

5. design of career development units 

Additional evaluations related to other components of the CPSS, 
i.e., resources assessment, annual program review and program le • 
assessment, were not attempted either because of difficulties ex- 
perienced in trying to design valid measurement schemes or because 
of the time limits prescribed by the study. CPSS requires three 
school years to install completely; the Assessment Study focused 
only on the first year projdircts of the use of CPSS. 

The results specified for Objective Two support a trend to- 
;vard higher quality career development activities in experimental 
schools. Of the eight statistical tests of change (4 ANOVA's and 
4 ANCOVA's) reported for the associated research hypothesis, four 
were significant at the p < .01 level and four were not signifi- 
cant at the p < .01 level. However, even in the instances where 
the findings were not significant, the direction of differences 
between experimental and control schools favored the research 
hypothesis (see Figure 4). 

Although the support for Objective Two is not as strong as 
that reported for Objectives One, la and lb, the evidence re- 
ported for all objectives supports the research hypotheses. 

The findings of this study were condensed into a Joint 
Dissemination Review Panel submission (see Appendix K) , a major 
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product of this assessment effort. The results of the Panel's 
evaluation of 'the submission were not available at the time of 
the final report publication. 
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APPENDIX A 
Dissemination Plan 



CAREER PLANNING SUPPORT SYSTEM (CPSS) 
ASSESSMENT STUDY 1978-79 

Dissemination Plan 

A3 specified in the Technical Proposal, p. 36, dissemination of CPSS and 
the results of the assessment study is an important outcome of the CPSS project. 
The following proposed dissemination plan has been prepared to meet the inten- 
tions for dissemination of CPSS presented in the Technical Proposal. 



DISSEMINATION PLAN 

Goals 

1. The CPSS package of materials will be reviewed and revised to correct 

minor printing errors and to include new information regarding the assess- 
ment study. 

2* Information regarding the assessment of CPSS will be presented in journals 
and at conferences; and information regarding the validity of iihe CPSS 
materials will be provided to guidance professionals (federal, state, 
local guidance staff, counselor educators, vocational educators and advi- 
sory groups). 

3. Each state department of guidance will receive a complimentary copy of the 
CPSS package. 

Objectives 

1. A. Each printed manual will be reviewed for typographical errors. New 

information about sites and a new forward will be prepared. The CPSS 
Decision Guide and Brochure will be updated. A new binder for the 
manuals and a container for the audio-visual materials will be designed. 



2. A. Federal, state and local guidance professionals (5,000) will be invited 
to participate in a pre-American Vocational Association (AVA) conven- 
tion workshop on CPSS on November 30, 1979 in Anaheim, California. 



Project staff and CPSS coordinators from three of the CPSS experimental 
schools will i^resent the results of the CPSS assessment study, review 
the CPSS procedural system and give advice to participants on securing 
funding for implementation of CPSS, including the National Diffusion 
Network. 

B. Journal articles will be written before the end of the project. Two 
types of articles have been drafted, one geared toward the practical 
aspects of the use of CPSS, i.e., what student needs were reported 
in the study, what factors impeded or enhanced the progress of CPSS 
in the school; and two, an article written for a research journal which 
describes the methodology for the evaluation of CPSS, e.g., instruments 
used, reliability, implications of rating procedures. 
, C. In addition to the pre-AVA convention workshop described in Objective 2A, 
an executive summary of the CPSS Assessment Study will be prepared 
and distributed to 5,000 guidance professionals and others throughout 
the U.S. by November 30, 1979. 
3. A. A complete package of the updated CPSS materials will be mailed to 



Methods 

The key audience will be those professionals associated with career 
guidance at the local, state, regional and federal levels. Others who will 



state guidance directors in all the states and territories by 



November 30, 1979. 
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be contacted are vocational education professionals, counselor educators, 
and school administrators. Other research and evaluation professionals may 
receive notification of project results throu^jh the normal dissemination 
procedures in operation at the National Center, i,e*, the National Center 
Clearinghouse, ERIC, and the Research Exchange. 

Exp e cted Ou t come s 

The experted outcomes of the dissemination plan are to report the practical 
usefulness of CPSS as a result of the national assessment. Analysis of the 
data is expected to provide evidence of the positive impact of CPSS in high 
schools. The purposes of the dissemination plan are to: 1) increase educators' 
awareness of CPSS and its benefits, 2) report JDRP approval of CPSS (provided 
approval is secured) , and 3) describe the methods used to assess the extent 
of planning for career development in a school setting and the possible effects 
of such planning on students. 
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CAREER DEVELOPMENT PROGRAM STATUS REPORT 



National Center for Research in 
Vocational Education 
1960 Kenny Road 
Columbus, Ohio 4321 0 



Introduction 

The career development program status report is a data collection instrument. Information will 
be collected from four distinct categories of data relative to a school's career development program 
planning: 

I Site Description— Community, school district and school 

II Career Development Program Description— staff organization, state and district goals, 
and an overview of activities and resources 

III Present Planning Efforts— personnel, assessment, ?^nd evaluation activities 

IV Projected Planning Efforts-personnel, assessment, and evaluation activities o 



Directions 

PLEASE TYPE ALL INFORMATION (N THE REPORT 

Please complete the report as fully as possible. Some questions request copies of documents, 
such as, goal lists, staff lists, brochures about your school's career development program, audiovisual 
materials lists, program plans or lesson plans. Whenever possible include copies of the documents in 
Jhe appropriate "attachments" section. For example, item no. 4 on page 3 refers to your school's 
course offerings and calendar. A copy of each should be included in the section marked "attachment 
1" because item no. 4 on page 3 is a part of Category I, SITE DESCRIPTION. The same procedure 
should be followed for any documents requested in the other categories: II, PROGRAM DESCRIP- 
TION; III, PRESENT PLANNING; IV, PROJECTED PLANNING. If there is no information a^-al- 
able for a question or section, please write N/A (not applicable) in the space. 

A National Center staff member will pick up and review the completed report during the 
Spring visit to your school. You and the National Center Staff member will review the report to- 
gether to insure completeness. 

If you have any questions about the report, please call James Pearsol, Program Associate, 
(614) 486 3655, ext. 406. Information included in this report will remain strictly confidential. 



CAREER DEVELOPMENT PROGRAM STATUS REPORT 



Please type all information in the report 



Name of School 
Address 



Zip Code 

Name of Principal 

Person Completing this Report 



Address (if different from above) 



Phone Number 



Position 

Date of Report Completion 



National Center for Research in 
Vocational Education 
1960 Kenny Road 
Columbus, Ohio 43210 
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CAREER DEVELOPMENT PROGRAM STATUS REPORT 



I. SITE DESCRIPTION 
A. Community 

1. To the best of your knowledge, identify the primary sources of income for residents 
of your attendance area by ranking the four most prevalent sources of income from 
the following list {l=most prevalent, 2-next highest, etc.): 

Professional, Technical, Managerial 

Clerical and Sales 

Service 

Farming, Fishery, Forestry and Related Occupations 

Processing _^ 

Machine Trades 

Benchwork 

Structural Work 

Miscellaneous 



Please Wst sources of information which your school has for general use which include descrip- 
tions of community resources available for career development activities. This should include 
such items as local post-secondary institutions, organizations available for job placement assist- 
ance, social or cultural service organizations, Federal agencies, business and industry, state and 
local agencies. 



School District 

1. The total enrollment of your school district is students. This figure 

represents students in grades — 

2. There are junior high or middle schools which feed into your high school. 

3. Please describe or attach information regarding career education goals for the feeder 
schools. 
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C. School 

1. Enrollment Number of Students 

Mate Female 

Grade 9 (if appropriate for your school) 

Grade 10 

Grade 11 

Grade 12 

Total 

2. Faculty/Staff Number 

Administrators 

Secretarial/Clerical 

Counselors 

Teachers 

Fulltime Specialists (please list) 



3. Part Time Support Personnel 

List the support personnel assigned to your school (such as psychologists, social workers, speech 
therapists, etc.), and the amount of time each spends in your school. 

Type No. of Days in School Duties 



4. Please attach a list of your school's course offerings and a 1978-79 school calendar. 



CAREER DEVELOPMENT PROGRAM DESCRIPTION 



A. Staff Organization 

Using the most current information you can find, show (1) the school district's organiza- 
tional chart and (2) your school's organizational chart, including lines of authority over- 
seeing guidance/career education activities. If it is not erearry shown on the organizational 
chart, provide a list of primary decision-makers and their positions and a list of committees 
involved with career guidance, if appropriate. 
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B. State and/or District Career Education Goals 



Attach, or write below, the career education goals for your state or district. Are these 
scribed or suggested? If there are none, please write N/A. 



C. Overview of Career Development Activities 

Please include as many copies of the next page as are necessary to fully, yet briefly, provide an 
overview of present career development activities in your school. One form should be com- 
pleted for each major activity (e.g,, event, instructional unit, group of activities or course). 
These should include both counselor and teacher-initiated activities. The following should 
help to clarify the type of information that is desired. 

1. Name of Activity: Use only if the activity has an official and/or popular designation that most 
people will recogni^e. If not, use Identifying Title. 

Identifying Title: 

2. Goal(s): List the stated goal{s) for the activity, if available, and how it was determined that the 
activity was appropriate for career development in your school. 

3. Objective(s): List the stated objectives for the activity, if available. 

4. Activity Leader: List the individual(s) within and, if appropriate, outside the school who has 
prime responsibility for the activity. 

5. Target Student Group: 

6. Location: Identify where the activity is being conducted. 

7. Schedule: Provide the approximate date(s) or time period(s) during which the activity is con- 
ducted and number of times presented. 

8. Activity: Briefly describe the activity. If possible, include methods, materials, and the amount 
of time needed by students to complete the method. 

9. Resources: Include the following, if appropriate and available. 

People: 

Materials: 

Equipment: 

Space: 

Funds: 

10. Evaluation: Brief!-/ describe any evidence that indicates that the activity is effective and appro- 
priate for continuoiion (both student evaluation and instructor evaluation). 



6 



NOTE: 



Make as many copies of 
this form as needed. 



CAREER DEVELOPMENT ACTIVITIES 

Please Type 

1. Name of Activity. 

2. Goal(s): 

3. Objective(s): 

4. Activity Leader{s): 

5. Target Student Group: 

6. Location: 

7. Schedule: 

8. Activity: 

9. Resources: 

People: 
Materials: 
Equipment: 
Space: 
Funds: 

10. Evaluation:^: 

WEm:^':^ 
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D. Overview of School Resources 



1. Career DevelopRi.'=5nt Print ":nd Audio Visual Materials (include catalog or listing if avail 
able)- Identify type of material and approximate number. 



2. Equipment: Identify types and number of audio visual equipment available for use in your 
career development program. 
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space: Identify any specific space that is available for career development activities* 



Funds: List the funds available for career developnnent activities; include sources, announts and 
any restrictions on their use* 



III. PLANNING EFFORTS FOR S'OUR SCHOOL'S CAREER DEVELOPMENT PROGRAM 
INITIATED DURING THE 1978-79 SCHOOL YEAR 



NOTE: Indicate N/A for any items which do not apply. 
A. Personnel Involvement in Planning 

Nature and Scope of Involvement 
(include amount of time involved 
Number and committee structure, if appro- 

Involved priate) 

Administrators 

Counselors 

Teachers 

Students 

Parents 

Community Representatives 
Others (please specify) 
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Assessment and Evaluation Activities 

1. Briefly describe or attach copies of any needs evaluation activities or surveys for career 
development which were conducted in the 1978-79 school year. Be sure to include who 
was surveyed, methods used, and what the results were. 
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Briefly describe or attach copies of any co.nniunity and school resource surveys which were 
conducted in the 1978 79 school year. List types of resources surveyed (i.e., people, equip- 
nnent, materials, space, funds, external services). 



Briefly describe any procedures which were used to review or evaluat^^ the 1978-79 career 
developnnent progrann at your school. 



Do you have specific career development program goals established for your high sch 
Yes No. If yes, answer the following questions. 

a. How were the career development program goals for this school year (1978-79) 
determined? 
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c. Were priorities established for the goals? 
How were goal priorities established? 



d. Were objectives for faculty/staff and students written for the goals? Yes No 

What was the procedure for writing objectives? 



EKLC 



14 



Indicate any change in career developnnent program activities for the 1979-80 school year 
which may have resulted from a review or evaluation of the 1978-79 career development 
program. 
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IV. PROJECTED PLANNING EFFORTS FOR YOUR SCHOOL'S CAREER DEVELOPMENT 
PROGRAM TO BE INITIATED DURING THE 1979-80 SCHOOL YEAR 

NOTE: Indicate N/A for any items which do not apply. 
A. Personnel Involvement in Planning 



Administrators 

Counselors 

Teachers 

Students 

Parents 

Community Representatives 
Others (please specify) 



Number 
Involved 



Nature and Scope of Involvement 
(include amount of time involved 
and committee structure if appro 
priate) 
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Assessment and Evaluation Activities 

1, Briefly describe any needs evaluation activities for your schoors career development pro- 
gram which are planned for the school year ( 1 979-80), including method and extent of 
evaluation. 



2. Briefly describe any community and school career development resource surveys planned 
for the school year (1979-80) and describe method and extent. 
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What process will be used for determining goais for the school's 1979-80 career developnnent 
program? 



Briefly describe any new career development program activities which you hope to develop 
during the coming year (1979-80) for implementation. How was it determined that these would 
be appropriate career development activities? 
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What plans, if any, do you have for reviewing or evaluating your career development program 
at the end of the school year (1979-80)? How will the review be conducted? 
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CLARIFICATION AND COMMENTS 

Please use this space to make any clarifying remarks, or to provide additional information about 
your school's career development program. Please indicate question nuniber if remarks pertain 
to a specific section of this report. Attach extra page(s) as needed. 
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AFPENDIX C 
Verification Checklist 




\'nc:i t i ona 1 Kduca t ion 
1^60 Kennv Road 
Columbus, Ohio ^3210 




VERIFICATION CHECKLIST 



Naniu of School 



Add rtvsrv 



Zip Code 



Naiiio of Principal 



(lu^rsou comp 1 •-•t: i n;', r(j[n)rt. ) Signature Bate 



Typed Name 



I liavr rt'vii'wed Lhc Lii To rina L ion in ttie checklist and verify the accuracy 
I) 1 I ht' i n lOniKi L ion , 



S i t i' Kt' n - SLMi La L i V I 



Si^^nature Date 



Typ^'^ Name 



Position 
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VERIFICATION CHECKLIST 



Directions 



Tho NntLonnl CeiUor staff member completing the form should base his/her 
rosponsfs on informal ion gn ined through documentation available at the site. 
Site records and Logs siiould be examined. Copies of documents such as ros- 
ters of committee members, schedules, and agendas should be obtained to sup- 
plement checklist responses. In addition, some interviewing of key personnel 
may be considered. 

It may be necessary for the staff member to exercise judgment about the 
accomplishment of planning activities at some of the sites. For example, in 
making judgments about the needs evaluation, it may be that a general needs 
assessment study was conchictetl for the sctiool or the school district. Wiiether 
or not Lhis study meets tho intent of a needs evaluation for career develop- 
ment depends upon sueh considerations as the recency of the study, the de- 
gree of its focus upon career development, and the extent to which the study 
conformed Co accepted needs evaluation procedures. 

The checklist provides space for comments to clarify checklist responses. 
The staff member should use this space to record information which supports 
his/her responses . 
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CO>t:-IlTTIi:ES 



1. Was a Career Development Program Planning 
Commit tee selected? 

2. Were selection criteria established for select- 
ing Career Development Program Planning Committ 
members? 

3. Was orientation to career development pro- 
gram planning provided for committee members? 

4» Did the committee meet on a regular basis 
throughout the planning effort? 

5. Did the committee coordinate the planning 
activities? 

6. Was an advisory committee organized? 

7. Were selection criteria established for 
selecting members of an advisory committee? 

8. Was orientation to career development pro- 
gram planning provided for advisory com- 
mittee members? 

9. Did the advisory committee meet on a regular 
basis? 

10. Were recommendations suggested by advisory 
committee incorporated into the program 
planning whenever feasible and practical? 

Commen ts ; 



1 
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IDENTIFICATION OF RESOURCES 



YES NO 

1. lias a committer been selected to identify re- 
sources? 



2. Was orientation to career development plan- 
ning; provided to committee members? 

■3. Have current career development activities 
been ident i f Led? 



4. Have school resources necessary for career 
development been identified? 

5. Have community resources necessary for 
career development been identified? 

6. Has a school and community description 
been i:)rejKired? 

7< Have available resources been matched with 
specific career development activities? 



Comments : 
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IDENTIFICATION 01^^ NEEDS 



YES NO 

1. Has a committee been selected to identify 

needs? 

2. Was orientation to career development pro- 
gram planning provided for committee mem- 
bers? 

3. Has a comprehensive needs evaluation for 

career development been conducted? 

a. Were students Surveyed? ^ 

b. Were faculty/staff surveyed? 

c* Were graduates surveyed? 

d. Were parents surveyed? 

4. Have needs data been tabulated and analyzed? ^ 

Comments : 
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GOAL SELECTION 



YES NO 



1. Was a comprehensive set of career development 
program goals devised as a result of identified 
needs? 

2. ' Were priorities assigned to goals based upon 

the identified needs? 

3. Were goals selected for implementation? 

4. Was a career development program goal review 
conducted? 



Comments : 



1^ p 
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CAREER DEVELOPMENT PROGRAM ACTIVITIES 



YES 

1. Wave potential career development activities been 
identified? 

2. Have instructors been selected? 

3. Have instructors been trained/prepared? 

4. Have program activities been written? 

5. Have program activities been reviewed and 

approved? 

6. Were program activities implemented? 

7. Were program activities evaluated? 



ComniLtnts : 



5 ^ J 



CAREER DEVELOPMENT PROGRAM REVIEW 



YES NO 



1. Has a general, annual review of Che career 
development program been conducted? 

2. Has the extent of i mplem e ntation of each 
career development program activity been 
determined? 

3. Have goals not yet implemented been reviewed 
for potential Implementation? 

4. Have decisions been made regarding program 
revision and/or expansion for the ensuing 
years? 



Comments : 
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Career Development Program Rating Instrument 

and Direc:tions 
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RATING INSTRUMENT 



Definitions and Directions 



Your task is to use .1 rating instrument to rate a school's career 
development prograT. i;i,.a r.- wru J ilirough two information references, "The 
Career Development Program Status Report" and "The Verification Chocklist" 
(defined below). Before you begin the rating tasks, READ THE FOLLOWING 

DEFINITIONS a:;d DiRi-:cTio:;s . ~ 

Definitions 

1. Reference - any information source describing a school's career development 
program which is identified and provided to the rater to answer an item on 
the rating instrument. 

2. S.R. - Career Development Program Status Report - a reference and 
information source, usually green in color. This document is a report 
generally prepared by school personnel describing the school career 
development program. 

3. V.C.L, - Verification Checklist - a reference and information source, blue 
in color. This document is a form prepared jointly by school personnel 
and project staff describing the school career development program. 

^. Career Development Activities - two specific career activities used as 
sources of information for raters (yellow cover sheet). 

^' Additional Career Activities - career activites used to answer the 
last item on the rating instrument (pink cover sheet). 



Direc tlons 

R EAD A LL T_HE_ I2^JpR>L\TJ0N_ PROVIDEp_ _FOR TH_E_ S CHOOL YOU ARE TO RATE BEFORE 
YOU USE THE RATING TNSTRU>1ENT__T0_ RATE" T^^^ This will familiarize 

you with the information available in the references. 

The rating instrument consists of 33 items and is divided into 7 
sections. Each section has directions which apply to the items in 
that section. When answering an item, place an "X" or"/' in the brackets 
[answer spaces] that in your judgment provide the best answer(s) to the 
rating items. 



(a) References are listed in the rating instrument either at the 
beginning of each section in tlie directions or following each item stem. 
The references listed are the only sources of Information to be used in 
rating an item. if a reference reads S.R. p. U-B, 1, it refers to the 



following : 
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S.R. = Career DovelopnonL Pro{>,ram StaLus Roport 

p. 11= p«u;«* 1 i'urn 

B, 1 = section H, nurnbc^r I 



If a reference reads VCL p. 3-1, it means: 

VCL = Verification Checklist - blue 
p. 3 = page uliree 
1 = number 1 

( b ) READ EACH RI^FRRFaXF: THOROL^GIILY BEFORE ANS\JERING THE R.\TING ITEM , 
Next to each reference will be a number or phrase (either "1", "2" or "no 
priority'*). The reference nunbered " 1 " should be given primary 
significance. For exanplo, [Rofuroncc: 1. SR p, 13-4, a; 2. VCL p, 
A-1]. Status Report paeo 13 nunber 4a should be considered most 
accurate if there is contiicting information. If there is jio 
conflicting information, use your best judgment based on the 
information referenced • 

In the instance when both references are of equal value, "no priority" 
is listed. This means that the rater should use all the information 
available to make the rating, with no special significance attached to 
either reference. 

(c) On the reference sources listed above, the Status Report (SR) and 
the Verification Checklist (VCL), the rater will often find 
"attachments'* listed. The rater should always consider the attachment as 
part of the reference when answering an item . For example, SR p. 5 might 
Include as part of a response, "See Attachment I." The rater should 

then turn to Attachment I, read it, and consider it part of SR p. 5 when 
answering the item. 

(d) In Sections IV and VII of the rating instrument, other references 
are listed either in place of or in addition to the Status Report (SR) and 
the Verification Checklist (VCL). In Section IV, the rater will use only 
the two attached career: development activities (yellow cover sheet). In 
Section VII, the rater will use all the references provided. (The 
directions for each section or item will alert the rater to special 
reference considerations. ) 

A. Conf id e nee Rating . After each item you will find a confidence racing. 
This rating is Lo be used to report the rater's confidence that the 
answer marked is correct. It reads as follows: 

CONFIDENCE RATING: Please estimate the probability that your answer is 
correct by placing a check at the appropriate location on the scale. 



I \ 1 H > 1 » 1 i i j 

0 10 20 30 40 50 60 70 80 90 100% 
no complete 



confidence 



con t 1^1 once 
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Goals and Objectives . When rat in;: :vH:cions II and III, whenever a 
reference uses the word "goal", consider it a goal. Whenever a 
reference uses the word "objective", consider it an objective, even 
though you may suspect that the person completing the form did not know 
the difference. 

The information listed in the references is the best information 
available to a.iswer each rating icera. Please use the information to 
make your best estimate even when you are uncertain, then use the 
confidence rating to express your uncertainty. 
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School y,o. 



CARblHR [)KVI:I.()1'MI:NT I'ROGRAM 
RATING INSTRUMENT 



Racer 

(numhor) 



I. Directions ; When answering items 1-4, consider the extent to which student 

needs for career development skills were idontified through a needs assessment. 

la. Was an assessment of student career development needs conducted? 
[ References : 1 . SR p. 11 ; 2. VCL p. 3-3 ] . 

[ ] No (0) IF NO. CHECK THE CONFIDENCE RATING AND SKIP 

TO SECTION II. p. 3 ...^ 

[ ] Yes (1) 



CONFIDENCE RATING: Please estimate the probability that your answer is 
ccrrccL by placing a check at the appropriate location on the scale. 

I- 1 1 1 1 1 \ 1 1 1 } 

0 10 20 30 40 50 60 70 80 90 100% 

complete 

confidence confidence 



lb. If yes, determine whether the needs assessment was: 
[ Referenc es: i^_^R_^_iJj_J^ J^CL_Pj._ >-3J . 

[ ] conducted only in the specific school (1) 

[ ] conducted in n larger unit, such as a school district (0) 



CONFIDENCE RATING: 

I j J , , J J. , j 

0 10 20 30 AO 50 60 70 80 90 100% 

complete 

confidence confidence 
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2. Wa ^; lIk' ur<'<|r; ,1 s sf !; Nf'irii t (liv;(j\ncd to a i st'Sis ^.tll(^•^L iumn! Iv^r career de- 
velt)pn<'nt skill:; or to d(»L(»nT)iuo sono o_t_lio r 1 n l^»rri.^t ijj_u , such as, a ncM»ils 
as:;o ssneu L ot eoimseiur roles iu career dc ve iopt u ;u 

[ ] Student centered (1) 

[ ] No n- s t ud e n t centered ( 0 ) 



CONFIDENCE RATING: Please estimate tlic probability that your mrsw. r is 
correct by pl.KMriy; n chock nr t.ho appropriate location an scale. 



1 \ \ \ \ \ 1 1 1 \ 1 

0 10 20 30 AO 50 60 70 80 90 100% 
no complete 
conf idence conf idence 



3. Were the results of the needs assessniont tabulated? 
[ ] Yes (1) 

[ ] No (0) IF NO , CHECK TUK CONKI DKNCE H^\TIN'G AND SKIP 

TO SECTION II, p, 3. 

CONFIDENCE RATING: 

I \ \ ! ^ \ h 

0 10 20 30 40 50 60 
no 

conf id e nee 



8u 91) 1(H):; 

con pi e te 
con 1 id eiice 



o 
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4. Uo tf rill i tic die r<ui>',c of rcspoiidcuics to the needs nsscssmeiit by checking 
below e.u'h of the r.roups chnt completed a questionnaire for the needs 
.isscssment . 



[ References 



h - ^C1'_Pj_J r 3_ a_,_ bj_ c, d; 2. SR p. 11] 

a. Students 

b. Faculty 

c. Recent graduates 

d. Parents 

e. Other 

f. None 



CONFIDENCE RATING: Please estimate the probability that 'your answer is 
correct by placing a check at the appropriate location on the scale. 



0 10 20 30 40 50 



-I 1~ 

60 70 



80 



90 



100% 



II, Direc_t;_ions : Wien answering items 5-7, consider the extent to which ^.oals^ 
were established for improving student career development skills. 

5. Are explicit goals for student career development reported? 
[Re/ e rences : (no_ priority) SR p. 5 and SR p. 13] . 

[ ] Yes (1) 

[ ] No (0) •'•^II_^9j_£33f'^_I^^_99IFlP^^?'^MF]r^9-^^^ 

SKIP TO SECTION III, p. 4. " 



CONFIDENCE RATING; 



0 

no 

confidence 



10 



20 



30 



40 



--I— 
50 



4- 



60 70 80 90 100% 

complete 
confidence 



-3- 
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6. Were, these cnroer developTr^ont gonls formulated from the res;ults of a 
needs nssessmnu ? 



[ ] Yes (1) 
[ ] No (0) 



IF NO, CHECK THE CONFIDENCE RATING AND SKIP TO 
SMCTlUN III, p, 4. 



CC)N'FIDE:;CK RAT[^:G: Please estinate the probability that your answer is 
correc:i: by placing a check at the appropriate location on the scale. 



10 



0 

no 

conf idence 



20 



30 



AO 



-t— 

50 



H- 
60 



H— 

70 



80 90 100% 

complete 
conf idence 



Were the reported goals for student career development organized in order 

of priority based upon identified needs? 

[References: (no priority) SR p. 14-c and VCL p. 4-2]. 



[ ] Yes (1) 
[ ] No (0) 



IF NO, CHECK THE CONFIDENCE RATIKG AND SKIP TO 
SECTION" III, p. 4. 



CONFIDENCE RATING: 



0 
no 

conf idence 



10 



20 



40 



50 



60 



70 



80 



90 



100% 
complete 
conf idence 



III. _p\ rec_l_i_ons : V/hen answering itens 8-10, consider the extent to which ob- 
jectives were written for student career development skills. 

8. Are explicit objectives for student career development reported? 
[ References: (no priority) SR p. 5 and SR p. 14-d ] . 



[ ] Yes (1) 
[ ] No (0) 



IF NO, CHECK THE CONFIDENCE RATING AND SKIP TO 
SECTION IV, p. 6 . — ^aenp. 



CONFIDENCE RATING: 



0 



10 



20 



30 



4U 



50 



60 



70 



yo 



90 



C OI.ip 1 i> t o 
CcmU idu'lK- 



ERIC 
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9. Use Che followinR references for items 9a-ci. 

( R e f c r e n c c sj (_i u )_ pjl o v_Liy) _ Sll_^ _ 5_ an d_ SR^ p_,_ ] • 



9a. Do most of the objectives state who are the key actors (e.g,, stu- 
dents, faculty)? 

[ ] Yes (1) 

[ ] No (0) 



CONFLDHNCt: RATINC;: Please estimate the probability that your answer is 
correct by placing a check at the appropriat e locat ion on the scale . 



0 
no 

confidence 



10 



20 30 40 50 60 70 80 



90 100% 
complete 
conf idence 



9b. Do most of the objectives state what behavior is to be demonstrated? 

[ ] Yes (1) 
[ ] No (0) 



CONFIDENCE RATING: 



0 

no 

confidence 



10 



— h- 

20 



30 



-1 

40 



50 60 70 80 90 100% 

complete 
confidence 



9c. Do raosu of the objectives state what are the ci5J'dJ._tJ._oris under which 
the behavior is to be demonstrated? 

[ ] Yes (1) 

[ ] No (0) 



CONFIDENCE RATING: 



I— +- 



0 
no 

conf i Jont L 



10 



1-. 

20 



30 



40 



50 



60 



70 



80 



1- 



90 100% 
compicLG 
conf idC'iiCL' 
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9d. Do most of the objectives state what de£rGe_ of su_ccess is required to 
achieve the objective? 



[ ] Yes (1) 
[ ] No (0) 



CONFIDEXCr: R^MIXC: Please estimate the probability that your answer is 
correct by placing a check at the appropriate location on the scale. 



I 4- t 

0 10 20 30 
no 

conf idence 



AO 



50 



1- 

60 







70 



80 



90 100% 

comple t e 
confidence 



10. 



Were the reported objectives written for, and designed to achieve 
specific student career development goals? REVIEW ITE MS 5- 9. 
[References : 



n o_ pri oj i ty )_ SR^ p. 1 3, S R p . 5, and SR p . lA-d ] . 
] Always (3) 

Mos t of the t ime (2) 
Occasionally (1) 
No (0) 



] 



CONFIDENCE 


RATING: 










f 1 

0 10 


20 30 


40 


50 


60 


70 80 90 100% 


no 










complete 


conf idence 










confidence 



Directions: Rate the attached career development activities (yellow only) 
on the criteria listed in items 11-20. If you have ^wo career develop- 
ment activities to rate, rate the first activity using column *'A'' and rate 
the second activity using column "B*\ If only on^ activity is attached. 



use column A 



If you have no activities to rate, check ^he b Qx be low ^ 

P-SKIP TO SECTipN_y, 



[ ] No career development activities attached 



11. Is there a goal li-SCed for the cnrecr developmc-nt nctivir.y? 



A B 

( ] [ ] Yes (1) 
[ ] [ ] No (0) 



IF. r'P_'. CjlKCK. T]iK_ CON FI_D1:_NCE_ R£Vr_I_NG_ 



CONFIDENCE RATING: Please estimate the probability that your answer is 
correct by pl.uing a check at the appropriate location on the scale. 



10 



10 



-I 1— 

30 40 



1 " 

50 



60 



70 



80 



90 



loo; 



B 



10 20 30 40 



0 
no 

confidence 



50 60 70 80 90 100% 

complete 
confidence 



12. Is the goal listed for the career development activity reported on SR 



lA? 




A 


B 


[ ] 


[ ] Yes (1) 


[ ] 


[ ] No (0) 



COXFIDEXCC RiMING: 

A , 

0 10 20 



— i- 
30 



— t- - 
40 



— 
50 



B 



---I 

10 



0 

no 

confidence 



-1 

20 



30 



1 



40 



50 



— j- 
60 



70 



-< i -1 

80 90 100% 



-t 1 * f 

60 70 80 90 100% 

compl ete 
confidence 
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13, Is an object ivi; (or ob j ec t: L vcs ) lisiud fur the career development' 
net Ivity? 



[ ] 
[ ] 



[ ] Yes (1) 

[ ] No (0) IF NO_^,CHECK TUE_ CONFIDENCE^ Jv\JJyp 

"^C^-.I^Rif- JP- J Ji-2l J 5_r pj. ip_ 



C0NFIDE::CE IL\Tr:C: pi ease estimate the probability that your answer is 
correct by placing a check at the appropriate location on the scale. 



0 10 20 30 



40 



-H 

50 60 



B 



0 

no 

confidence 



\ \ ^ 1 ^ — 

10 20 30 40 50 



"1- 



60 



70 



80 



H 



70 



90 



100% 



80 90 100% 

complete 
conf idence 



14. Answer the following questions (14a-d) for activity A in the coluipn marked A 
and for activity B in the column marked B. 

14a. Do most of the objectives listed in the activity state who are the 
key ac_tjDrs (e.g. , stude.^us, faculty) ? 

_A B_ 

[ ] [ ] Yes (1) 

[ ] [ ] No (0) 



CONFIDENCE RATING: 



r 



_ I — 
10 



10 



0 
no 



20 

- 1 — 

20 



■* - 

30 



30 



~ < — 

40 



50 



50 



60 



60 



_» 



70 



~t 

70 



1— 

80 90 



— f-- 

80 



100% 



-f 



90 100% 

complete 
con t" i JiM) ; 
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I4b. Do most of tlie object ivos listed in the activity state what behavjpr 
is to he demonstrated? 



_ A 

[ ] [ ] Yes (1) 
[ ] [ ] No (0) 



CONE'^IlJiLNCl:: RiXTINC: Please estimate the probability that your answer is 

correct by placing a check at the appropriate location on the scale. 

0 10 20 30 40 50 60 70 80 90 100% 

0 10 20 30 40 50 60 70 80 90 100% 

no complete 

confidence conf idence 



14c. Do most of the objectives listed in the activity state what are 
the _condit_p,_on_s under which the behavior is to be demonstrated? 

_ A_ 

[I [ ] Yes (1) 
[ ] [ ] No (0) 



CONFIDENCE RATING: 
A 

0 10 20 30 40 50 60 70 80 90 100% 

0 10 20 30 AO 50 60 70 80 90 100% 
no complete 
confidence confidence 



ERIC 
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Do most of the ob.j c-cl i vcs listed in the ncCivity state what der,re_a 
o\' siu'C'rss is rffjuirt'd to acliic*ve objectives? 





A 


B 




[ J 


[ J \cs (1) 




[ 1 


( ] No (0) 


CONFIDENCE 


ii^vriNc;: 


E^lcasc estimate the probability that your answer is 


correct by 


p I ac ing 


a clieck at the appropriate location on the scale. 


A 






0 10 




/■ n An 7n ftn on i nn^ 
jU I)U du /u ou xuua 


B 












0 10 


20 


30 40 50 60 70 80 90 100% 


no 




complece 


confidence 




confidence 



Was a target student group reported for the activity? (e.g., the 284 
members of the junior class, or the 77 sophomores in Biology 100) 

_ A_ _B 

[ ] [ ] Yes (1) 
[ ] [ ] No (0) 



CONFIDENCE 


R.MING: 


















A . 












i — 






1 


0 10 

^ H- ^- 


20 


--^ 

30 


- f 

AO 


50 


60 


70 


80 

1 — 




90 


100% 
1 


0 10 
no 

confidence 


20 


— , 

30 


— ♦- — 

AO 


50 


— f- — 

60 


70 


80 


1 — 

90 


100% 
complete 
confidence 
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16* Were explicit mc^tliods for instruction reported for the career development 
activity? 

_ A B_ 

[ ] [ ] Yes (1) 

[ 1 [ ] No (0) 



COiNFIDENCE JIATING: Please estimate the probability that your answer is 
correct by placing a check at the appropriate location on the scale, 

0 10 20 30 40 50 60 70 80 90 100% 

B ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 

0 10 20 30 40 50 60 70 80 90 100% 
no complete 
confidence confidence 



17* Is there reported evidence that the effects of the career development 
activity are determined by means of a student outcomes measure? 



_ B_ 

[ ] [ ] Yes (1) 

[ ] [ ] No (0) 



CONFIDENCE RATING: 

A ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 

0 10 20 30 40 50 60 70 80 90 100% 

B ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ 

0 10 20 30 40 50 60 70 80 90 100% 

no complete 

confidence confidence 
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18. ll.l^;ud uii your rxju' r i tMifi' , wli.it is Lhc (h.-iine ihnt l]\c methods doscrLbcd 
for v:\c]\ tif till' act. i vi I ii'S will achieve i]\c specific ob/ioctivos listed 
for the acL iv icy? \y\ .YOI?_ MAUKj:!) ^'Np"_ 1\\ KmiKR 1 Tl-.M^ \}_ OR ]JK>1 1 6_, 
MAKK "NOT AI'rLiCAlU.V; " AlA) SK i V" "I'o' Vl LM 19 * ' ' ' ' 



Activity A: 
Activity B: 



[ ] 

[ ] 
Good 
Chance 



a:.\)^ SKj 
[ ] 

[ ] 
Some 
Chance 



[ ] 

[ ] 

Little 
Chance 



[ ] 



[ ] 



[ ] [ ] 

Vciry Little Not 

Chance Applicable 



CONFIDENCE 
correct by 


R/\TING: 

plae. ing 


Please estimate the probability that your answer is • 
a check at the appropriate location on the scale* 


A 








^ 4 1^- 




0 10 


20 


30 40 


50 60 


70 80 90 


100% 


B 






1 ^. 






1 \ — 








• 


0 10 
no 

confidence 


20 


30 40 


50 60 


70 80 90 


100% 
complete 
confidence 



19. What is your best estimate of the chance that each of the career 

development activities will improve student career development skills, 
irrespective of whether the career development skill is listed as an 
objective of the activity. 



Activity A: [ ] 

Activity B: [ ] 
Good 
Chance 



[ ] 
[ ] 

Some 
Chance 



[ ] 

[ ] 

Little 
Chance 



[ ] 

[ ] 

Very Little 
Chance 



CONFIDENCE 

A — 


RATING: 




40 






70 








0 10 
B ^ ^ 


20 


-V - 
30 


50 


1 — 

60 


-1- 

80 


1 — 

90 


# 

100% 
4 


0 10 
no 

confidence 


20 


-1 — 
30 


40 


, 

50 


— — 1 — 

60 


70 


— » 

80 


1— 

90 


100% 
complete 
confidence 
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10. 



Based on your experiences with the career developmenc needs of high 
school students, does each activity meet a student career develop- 
ment need? 



[ ] [ ] Yes (i) 

[ ] [ ] N'o CO) 



CONFIDENCE RATING: Please estimate the probability that your answer is 
correct by placing a check at the appropriate location on the scale. 



4— 

0 



B 



0 
no 

confidence 



10 

- 1- - 
10 



~» 

20 30 



40 



50 60 



--r 1— 



-t- 



-I \ 

20 30 40 50 60 



70 



70 



80 90 



100% 



-t 



80 90 100% 

complete 
confidence 



Is each activity constructed to permit easy use by faculty and staff? 

__A B__ 

[ ] [ ] Yes (1) 

[ ] [ ] No (0) 



CONFIDENCE RATING: 

A . ... 

0 10 20 

B ^ ^ ^ 

0 10 20 
no 

con f idencG 



1- 

30 40 



30 



f. 



50 



40 50 



60 70 



60 



— « — 

80 



70 



80 



90 



— ♦ 

100% 



90 



100% 
complete 
conf idonce 



Ts onch nc L i v i ty c 


onnt rii( 


•tod to cntn>a r.'ir.e student accoptanco? 


A 


B 








[ ] 

[ ] 


[ ] 
[ ] 


Yes (1) 

No (Q^ 






CO \ F 1 D f ' \ C I' liA'V I i\' C\ ' 


I^leasc estimate 


the probability that your answer is 


ci^rrecL by placint; 




:k aL the appropriate location 




A 










0 10 20 


30 


40 50 


60 70 80 


1 1 

90 100% 


B 












30 


40 50 


60 70 80 


90 100% 


no 

conf i donee 








complete 
confidence 



JP^i?-?-? * Items 23-26 refer to staff organization for the career develop- 
ment program. Rate the items as presented. 

23. Was a committee selected to plan a program for student career development 
[ ?^f'l9IS^PSP?:' _ . - y C_L p_. _ l_-_lj ^ 2^ ^ S R_ Pj^^ 1 OJ . 



[ ] Yes (1) 

[ ] No (0) «^ IF NO, CHECK THE CONFIDENCE RATING 

Ai^D SKIP TO ITEM 26. p. l5 -li^^* 



CONFIDENCE 


RATING: 




t ^ 




— 1 i 


0 10 


20 30 40 50 60 70 80 


90 100% 


no 




complete 


confidence 




conf idence 



24. Did the committee meet regularly during the planning period? 
til?/ eren_cc_s_:_ _ 1 VCL p_._ _l_-4j_ 2_^._ SR_2_.^J.O] . 

[ ] Yes (1) 

[ ] No (0) 



CONFIDENCE RATING: 



f 



t r 1 



0 10 20 30 40 50 GO 70 80 90 lOOZ 
ronl idi iu o • conf icU noo 



Did thcj cc^rnin i t t.u'c c:ot> rd injU c pJanning activiLies? 





^: ] . VCI. j). 1- 


2. SR p. 10] . 




[ ] Yes (1) 






f ] No (0) 




co>:fide:ncE': 

correct: by 


RATT::G: Please 
placing a check 


cscimate tlie probability that your answer is 
at the appropriate location on the scale . 


0 10 
no 

con f idence 


20 30 AO 


50 60 70 80 90 100% 

complete 
confidence 



^gi:^_I5t_ing_items_26a_- g, _ refer ba ck to IT EMS 2 1-23: V CL p, U_niimb£r,s, 
i^_2_,_3j.^i_§nd_5i_aDd_SR_p. . iO_ Ino_priQri£:y ) . 

26a. Were school administrators involved in planning the career develop- 
ment program? 

[ ] Yes (1) 

[ ] No (0) 



CONTIDE\^CE 


IhVl' 1 XG : 














0 10 
no 

con f idence 


20 30 


40 


50 


60 


70 


80 


90 100% 

complete 
confidence 



If "Yes", rate the extent of involvement; 



[ ] 

High 
(3) 



[ ] 
Moderate 
(2) 



[ ] [ ] 

Low Negligible 

(1) (0) 



[ ] 

Cannot 
Determine 
(9) 
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2(A), UVt(j sclioul r Dim S' • I <) n; Ihvc)Iv(;h1 in [il.iiniin}; t cai-cc'i* dove lo|)nLMit proj;raii? 

I 1 Yc.s (I) 
t J No (U) 



CONFI DK.\'CK R/\TINf.;: Plonse cscim.ice the probability that your answer is 
corroct by placing a check at tho appropriate location on the scale. 



I ^ 



0 
no 

conf idence 



10 



H 1 1 



3U ^'J 50 60 70 80 



90 loo:: 

complete 
confidence 



If "Yes", rate the extent of involvement: 



[ ] 

n±oh 

(3) 



[ ] 

Moderate 
(2) 



[ ] 
Low 

(1) 



[ ] t ] 

Negligible Cannot 
(0) Determine 
(9) 



26c* Were school teachers involved in planning the career development program? 

[ ] Yes (1) 
[ ] No (0) 



CONFIDENCE RATING: 



0 
no 

confidence 



10 20 30 AO 50 60 70 



80 90 100% 
complete 
conf idence 



If "Yes*\ rate the extent of involvement: 



[ ] 

High 
(3) 



[ ] 

Moderate 

(2) 



[ ] 

Low 

(1) 



[ ] 

Negl igible 
(0) 



t ] 

Cannot 
Determine 
(9) 
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26d. Were school studeaLs Invoivod in pinnainjj the career deveiopnieat progran? 

[ ] Yes (i) 
[ ) No (0) 



^ "FIDENCE RATING: Please estimate the probability that your answer is 
correct by placing a check at the appropri^itc location on the scale. 



0 
no 

conf idence 



■i \ i i \ \ 1 1 1 

10 20 30 40 50 60 70 80 90 100% 

complete 
conf idence 



26e. 



If "Yes", rate the extent of involvement: 

[ ] [ ] [ ] [ ] [ ] 

High Moderate Low Negligible Cannot 

(3) (2) (1) (0) Determine 

(9) 

Were community aie.abers involved in planning the career development program 
[ ] Yes (1) 
[ ] No (0) 



CONFIDENCE RATING: 



0 10 

no 

conf idence 



20 



30 



AO 



50 



4- 



70 80 90 100% 

complete 
confidence 



If "Yes", rate the extent of involvement 



[ ] 
High 

(3) 



[ ] 
Moderate 
(2) 



[ ] 
Low 

(1) 



[ ] 

Negligible 
(0) 



[ ] 

Cannot 
Determine 
(9) 
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26f . Wore partMiLs i[ivoLvt'il In pLanniiu; the. career devcilopnicMit j)rogram? 

L ] Yes (i) 
[ 1 No (0) 



CONFIDENCE RATING: Please estimate the probability that your answer is 
correct: by placing n check at the appropriate location on the scale. 

I 1 \ 1 \ i -1 \ 1 1 1 

0 10 20 30 40 50 60 70 80 90 100% 
no complete 
conf idence conf i^lence 



If "Yes", rate the extent of involvement: 

[ ] [ 1 [ ] [ ] [ 1 

High Moderate Low Negligible Cannot 

(3) (2) (1) (0) Determine 

(9) 

26g. Were others Involved in planning the career development program? 

[ ] Yes (1) 
[ ] No (0) 



CONFIDENCE RATING: 



0 

no 

confidence 



10 



-i 

20 



30 



AO 



— 1 — 
50 



4- 



60 70 80 90 100% 

complete 
conf Idence 



If "Yes", rate the extent of involvement; 



( 1 

High 

(3) 



[ ] 

Moderate 
(2) 



[ ] 

Low 

(1) 



[ ] 

Negligible 
(0) 



[ ] 

Cannot 
Determine 

C9) 
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VI. Directions : Item 27 is a probability estimate of the impact of future plans 

for the career development program in the school. Rate the item as presented. 



27. If Che plans reporCed on SR pp. 16-20 are carried out, what, in your ex- 
perience, is tlie chance that student career development skills will be 
improved? 

[ ] [ ] [1 [ ] 

Good Some Little Very Little 

Cliance Cliance Chance Chance 



CONFIDKNCK RATING: Please estimate the probability that your answer is 
correct- by placing a check at the appropriate location on the scale. 

( \ 1 , ^ ^ 1 , 1 

0 10 20 30 40 50 60 70 80 90 100% 

no complete 
confidence confidence 



VII. Directions : After completing the previous items, you probably will have 
formed judgments about the overall career development program of the 
school. Use SR pp. 5 and 10-20, VCL pp. 1-3 and 4, and all previous 
items to rate tlie following items except item 32. Item 32 requires you 
to use only the yellow career development activities. Place a check in 
the response space that best represents your judgment on each item. 

::8. Estimate Llie extent to which th^ school staff was organized to plan sys- 
tematically a comprehensive career development program by evidence of 
clearly des i-;na itul leadorshi;); adnlnistra tive cooperation; and permanent, 
active groups and commit tees. 



[ ] [ ] 

(0) (1) 
Limited Extent 



[ J 
(2) 



[ ] [ ] 

(3) (4) 

Great Extent 



CONFIDENCE RATING: 



-f- 



0 
no 

confidence 



10 



20 30 40 50 60 70 80 90 100% 

complete 
confidence 
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29. KsLiiiKiLo lIr' c'XLlmU to whicli a sLucIlmU career dcveiopmcnt needs as- 
SOS sment w:j:; conducLcd , tabulated, properly interpreted, and the data 
uLilized for plaiuiini; the career dove 1 opnen t pro^^ram. 



[ ] [ 1 

(0) (1) 
LiniitiC'i. F.NCunL 



[ ] 
(2) 



I ] 
(3) 



[ ] 

(A) 

Great Extent 



CONFIDENCE RATING: Please estimate the probability that your answer is 
cnrrocc by pl.iring :i rhc^k nr the appropriate location on the scale. 



■I 



0 10 20 30 40 50 60 70 80 90 100% 
- no complete 
confidence confidence 



30. Estimate the extent to which a comprehensive set of ordered career de- 
velopment goals reflecting assessed student career development needs were 
developed and used in planning, implementation and evaluation of the 
program • 



[ ] [ 1 

(0) (1) 
Limited Extent 



[ ] 
(2) 



[ ] [ ] 

(3) (A) 

Great Extent 



CONFIDENCE RATING: 



4- 



0 
no 

confidence 



10 



20 30 40 50 60 70 80 90 100% 

complete 
confidence 



31. Estimate the extent to which a set of behavioral objectives was developed 
reflecting specific goals and containing a clear statement of the intended 
audience, behavior, situation and standard of mastery. 

[ ] [ 1 [ 1 [1 [ ] 

(0) (1) (2) (3) (A) 

Limited Extent Great Extent 



CONFIDENCE RATING; 



I) 10 2U 3U -'.U 5U 6U 70 8U 90 100,: 

no coinplt'to 
cDut id(.-iu'o con r idiMu; <: 

O — 



32. Estimate the cmLlmU to which career development £u:tivities v;ore devMoped 
that reflect studoiit noeds, p.onls, and associated objectives, and that 
IndiciiLe t.icthfMis, L<ii\;ct student yroup and outcone neasures by referring to 
the two attachi/d caree r develo[':.ien t activities (yellow) > 



[ ] I 1 

(0) (1) 
Limi ted Lxt ent 



[ ] 

(2) 



[ ] 
(3) 



[ ] 
(A) 

Great Ex 



tent 



CONF IDENCK RATING: PieasG estimnte the probability that your answer is 
corroci by p!.i--inf; n chcrk nc the npproprinte location on the scale. 



0 
no 

conf idence 



10 



20 



30 



40 



50 



— I — 
60 



-1— 
70 



— t — 
80 



90 100% 
complete 
confidence 



33. 



Based on the available information (including all yellow and pink career de-~ 
velopment activities), rate the overall quality of the school ' s career develop- 
ment program. 



[1 [ ] 

(U) (1) 
Very Low Quality 



(2) 



[ ] [ ] 

(3) (4) 

Very High Quality 



CONFIDENCE RATING: 



0 

no 

conf idence 



10 20 30 40 50 60 70 8U 90 100% 

conple te 
confidence 
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Career Planning Support System Knowledge Test 



124 -^^0 

O 

ERIC 



Test 2 



Career Planning Support System 
Knowledge Test 
for 

Steering Committee Members 



Instructions : Now that you have taken an attitude survey, we would like 
to assess your technical knowledge of the activities involved in planning 
for career development. 

This test can only measure the approximate extent of your know- 
ledge due to differences in your educational experiences and interests. 

Do not write your name on the test. All tests will be sealed in 
an envelope immediately after the test and mailed to Ohio State University. 
Your individual anonymity is guaranteed. 




Steering Committee Knowledge Test 



1. Planning a career development program should take Into account the 

available resources which Include peoj)le, programs, materials, equip- 
ment, and funds. State why it is important to assess each resotirce 
with one sentence. 



a • 


People 


b. 


Programs 


c • 


Materials 


d. 


Equipment 


e. 


Funds 



2, Performance objectives are a key ingredient in career development 
instructional units. What is a performance objective? 

3. Define "resource accounting" in the S(»nse the term is used in career 
development program planning, 



A, When should continuously used career development units be evaluated' 



5, Mention five ways in which students can be involved in planning a 
career development program. 

a. 



d. 



e. 



6. One method of achieving a career development program goal is to develop 
an instructional unit that aids students in attaining the goal. List 
the components of a comprehensive career development Instructional unit. 
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?• A school's career development program Hhould be reviewed annually. 
State five criteria that should be usi?d in this evaluation. 



8, Which groups of people should be involved In evaluating the success 
of a career development activity? 



9» What function does the statement of the performance objectives serve 
in the construction of a career development activity? 



10. Conducting a needs assessment will yield information that is useful 

in developing a career development program. Describe the needs assess^ 
ment procedures which you think would be appropriate for your school. 
Mention who should be assessed, when the assessment should occur, what 
should be assessed and how results might be used. 



11. List five criteria on which goals for career development programs might 
be arranged In priority order. 



a. 
b. 



e. 
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12. After goals have been established for the career development program 
they should be arranged In a priority order. State four reasons why 
emphasizing goals according to priority is more desirable than emphaslzinR 
all goals equally* 

a. 

b. 

c . 

d. 



13. A school which assesses community resources should coordinate their 
assessment with that being done by other schools. Describe three 
benefits this might provide. 

a. 

b. 



14. Although instructional units for career development are written primarily 
for students* teachers find them to be very useful. Cite two ways in 
which teachers might find the units to be valuable, 

a. 



b. 



15- Which school staff members should Implement a career development unit? 



16. I.ist three criteria that are used for selecting career development goals 
a. 
b. 



c • 
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17, Surveying graduates can yield information useful In planning a school's 
career development program. Which fcnir kinds of information do you 
think are most important to collect from the graduates (and those who 
didn't graduate) and how could each be useful? 

a. 

b. 

c . 

d. 



18. Explain the advai.tat.es or disadvantages of assessing the needs of students 
by studyiap: a representative sample of student from all grades compared to 
studying a specific group of students such as sophomores* potential 
dropouts, studeni . taking biology, etc. 



19. How often in a slx^year period should a needs assessment survey of 

studcnta be conducted? Give three arguments in support of your answer. 

1. 



2. 



3. 

20« List and describ{» ur uses for the roHults of an annual review of a 
school's career dtwelopment program. 

1. 

2. 

4. 



21. A. In your opinion, who should participate in writing the behavioral 
objectives for a career development unit? 



1 ^ - 

Scnte .'It lea.'U cwo reasons for each person listed above. 




5 



22. Give three examples of how resource oHBessment results might be used In 
planning a career development program and putting it into effect. 



23. Wliar four criteria would you recommend to assoss the effectiveness of 
a career development unit? 



b. 



24. State five goals that are typically set for a school's career develop- 
ment program. 



e . 



25. State four bases for identifying goals for a ochool's career develop- 
ment program. 

a. 

b. 

c - 



26. Newly prepared career development units might contain imperfections 

which should be corrected before the unit is made available for routine 
uae» In your opinion, name the categories of people who should partlct* 
pate in the preliminary evaluation of such a unit. 
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27. Please respond briefly to A or B: 

A. For each of the pro^'^ram phases randomly listed below, describe 
in each column who should be involved, the order in which each 
should occur, and tht» Intended purpose. 



Program P hawses 

faculty/staff impiementat ion 

needs assessment 

evaluation of the effectiveness 
of each career development unit 

program review 

setting goals and performance 
obj ect ives 

resource account in^ 

writing career development 
units 



Who 



When 



Purpo ee^ 



B. For each of the program phases randomly listed below, describe 
in each column a potential problem and practical remedy/advice 
you would give . 



Program Phases 

faculty/staff imp I ementat ion 

needs assesflmcnt 

evaluation of the effectiveness 
of each career deve Lopment unit 

program review 

setting goals and performance 
object ives 

resource accounting 

writing career development 
un i L*s 



Problem 



or Advice 
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inn uiHO sia«« UnivArsiiy ' 1900 Kf,nny Road • Cotumhtit Ohio 432K) 
U\ (0141 486 3656 Culila CTVOCrOOSU/CnUimlnii Ohm 



Dear Steering Committee Member: 

We are pleased to have your participation in the CPSS assessment 
project. Your involvement with CPSS demonstrates your commitment to the 
development of career planning In your st:hnoL. In addition, your contri- 
bution is a vital part t)f a nationwide research effort. 

The following is a Lwo-part test. The first section assesses your 
per<'eptions o[ p J aiming for c areer dev«»lopment and the second assesses 
your technical knowleti^e of the activities Involved in planning for 
career deveIo[)men t . 

Yo u will have A3 minutes to complete these tests. Because these 
are timed tests, please work quickly. If you do not know an answer, 
simply on to the next question. You may look back over any unanswered 
Items after you liave completed all the questions. 

Individual scores will not be used or reported. Your test will be 
sealed in an envelope when you are finished to insure that your answers 
remain anonymous. We sincerely thank you for your cooperation. 



S Incerely , 




Donald C. Flndlay 
Project Director 
CPSS Assessment Study 
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PKRCEPTIONS OK PK()<;i<AM PLANNlNfJ FOR 
CARREK DiiVI'.l.OPMENT 



01 rec tlon s : Hiisud on your [)c»rcept J ons of plaiiiilnj^ for career pro^;rams, 
will you pleaBc indit:aLe your Leviil of agreement with each question. 
There are no "right" t)r "wronjj;" answers on this first evaluation. 
Please use this scale to express your attitudes toward statements 1 
through 1 7. 













(U .-H 


(U 








U 00 


u 






00 


00 c 


00 


u 




C Oi 


03 O 


<0 


LJ 


(U 


O (U 


U) U 


Ui 






»-« >^ 


-H iJ 






00 


4-1 6r 




Q 




<: 




I)S 


I) 


N 


A 


SA 



For Instance, If you strongly agree with a statement, you would circle 
SA (stronKly a>;ree) . 



PERCEPTIONS OF PROGRAM PLANNING 
FOR CAREER DEVELOPMENT 



Plans for career deveLopment programs 
are beat made by teachers, counselors, and 
career speciaLists, working separately 
from each other. 

Information which couLd be supplied by 
graduates has little use for career program 
planners because of its datedness* 

Knowing parent attitudes about career develop- 
ment is desirable when planning local 
programs . 

Schools need to focus more effort on 
planning for career deve lopment before 
putting the program into effect. 

The career development needs of students 
should be met entirely by counselors. 

Industry, labor, and the professions should 
work hand-in-hand with schools in planning 
career development programs. 




7. Business, industry^ and labor naturally 
induct students into careers; schools 
need no t be concerned about it. 

8. Career development is a highly specialized 

and critical part of the total school curriculum 
and should be given appropriate priority, 
staf f ingy and reserves . 

9. In-school career development activities 
should be conducted independently of 
the community. Otherwise, career 
aspirations of students will be unduly 
influenced . 

10. Since the career development of youth is 
a universal problem, no special local 
information about local student status is 
necessary when planning individual school 
programs • 

11. Schools spend so much time planning for career 
development that it hinders implementing 

the program. 



12. "Subject matter teachers" should 

cooperatively participate In planning, 
developing, and Implementing career 
development act Ivl ties. 



13. Developing units of instruction for career 
development requires too many special akil 
to be done by the average teacher. 



1^, The goals for a career development program 
should be based largely on the identified 
needs of students. 



15. It is sufficient to review a school's 
career development program once every 
five years. 

16. Career development program goals should 

be arranged in priority to improve chances 
of achieving them. 



17. Limiting career development activities to 
community opportunities is a good idea 
because otherwise students would have 
unrealistic aspirations. 
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B. Directions s Please use the following key to complete items 18 to 32. 



>> 
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U 
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> 

VL 



If comprehensive, systematic procedures for planning the career development 
program were instituted at your school, how likely is it that such planning 
procedures would: 



18. result in better relationships among 
faculty/staff and students? 
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19. result In more efficient use of resources 
for career development? 



VU U N L VL 



20. improve present career development planning 
procedures 7 



VU U N L VL 



21. have little potential for directing studentj 
toward worthwhile career goals? 



VU U N L VL 



22. result in better student planning for career 
development? 



VU U N L VL 



23. better meet student needn? 
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24. result in more relevant career 
development program objectives? 















1-4 
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25. Increase faculty/staff support for a career 
development program? 



VU U 



N 



VL 



26. require more work than can be handled by 
the existing faculty/staff? 



VU U 



N 



VL 



27. identify goals for a career deveLopment 
program? 



VU u 



N 



VL 



28. make me perceive career development 
more positively? 



VU U 



N 



VL 



29. Identify student needs which were not 
apparent before? 



VU U 



N 



VL 



30. result in an annual career development 
program review? 



VU U N 



VL 



31. result in narrower participation in the 
icapLementation of the career development 
program? 



VU U N 



VL 
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32. result in biased or stereotyped perceptions 

If: *w 
J . > 



VU U N 



VL 
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CPSS ASSESSMENT STUDY 
REVIEW PANEL 



Dr. Henry Borow 
University of Minnesota 
106 Nicholson Hall 
Minneapolis, MN 55455 

Dr. John Crites 
10897 Deborah Dr. 
Potomac, MD 2 08 54 

Dr. William Erpenbach 
Wisconsin Dept. of Public 

Instruction 
12 6 Langdon St- 
Madison, WI 53702 

Dr. Larry Gulick 
1578 Chippewa Ct. 
Grove City, OH 43123 

Dr. David Jesser 

1872 South Pierson Ct. 

Lakewood, CO 8 0226 

Ms. Elaine Jones 

Ft. Steilacoom Community 

College 
9401 Farwest Dr. 
Tacoma, WA 9 8498 



Dr. G. Brian Jones 
American Institutes 

Research 
1791 Anastradero Rd 
P.O. Box 1113 
Palo Alto, CA 94302 



for 



Dr. Juliet Miller 
Project Director 
Rehab. Research Institute 
School of Education 
University of Michigan 
Ann Arbor, MI 48109 

Dr. Clayton Omvig 
Dept. of Vocational Education 
Room 1 5 , Dickey Hal 1 
University of Kentucky 
Lexington, KY 40506 



Dr. Samuel H. 
575 Enfield 
Columbus, OH 



Osipow 



43209 



Dr. Alice Scales 
4H01 Forbes Quadrangle 
University of Pittsburgh 
Pittsburgh, PA 15260 

Dr . Bruce Shertzer 
Education Building 
Purdue University 
W. Lafayette, IN 47907 

Ms. Jessie Teddlie 
1952 Yucca Trail 
Hurst, TX 



Dr. Robert Lathrop 
Florida State University 
Tallahassee, FL 32306 

Ms. Nancy Losekamp 
1950 N. Mallway 
Columbus, OH 4 3221 
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ORIGINAL TARGET CITIES 



ARIZONA 
Phoenix 
Tucson 

CALIFORNIA 
San Diego 
San Jose 

COLORADO 
Denver 

DISTRICT OF COLUMBIA 
Washington 

FLORIDA 
Miami 

GEORGIA 
Atlanta 

ILLINOIS 
Chicago 

INDIANA 

Indianapolis 

KENTUCKY 

Louisville 

LOUISIANA 
New Orleans 

MARYLAND 
Baltimore 

MICHIGAN 
Detroit 

MINNESOTA 

Minneapolis 

MISSOURI 

Kansas City 
St. Louis 



NEBRASKA 
Omaha 

NEW JERSEY 
Newark 

NEW YORK 

Brooklyn 
Buffalo 

NORTH CAROLINA 
Charlotte 

OHIO 

Cincinnati 
Toledo 

OKLAHOMA 

Oklahoma City 
Tulsa 

OREGON 

Portland 

PENNSYLVANIA 

Philadelphia 
Pittsburgh 

TENNESSEE 
Memphis 

TEXAS 

Dallas 
Houston 

UTAH 

Salt Lake City 

WASHINGTON 
Seattle 

WISCONSIN 

Milwaukee 
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THE NATIONAL CENTER 
FOR RESEARCH IN VOCATIONAL EDUCATION 

ThB OhKi SI.IIK UnivOfSily • I960 Kennv Roari • Co: ..mbi.!., Oh.o 43210 
Tel (6141 486 3655 Ciitjlu CTVGCEDOSU/Columbus, Ohio 



July 21, 1978 



Dr. M. D. Thomas 
Superintendent of Schools 
Salt Lake City School District 
440 East First South 
Salt Lake City, Utah 8411 1 



Dear Dr. Thomas: 

We at the National Center for Research in Vocational Education are interested in identifying 
a set of high schools to participate in a controlled assessment study during the 1978-79 school year 
The product to be tested is the Career Planning Support System (CPSS), a systems approach to 
upgradmg career guidance services. CPSS is a comprehensive career guidance development tool 
that emphasizes program planning, implementation and evaluation. A further description of CPSS 
will be found in the CPSS Decision Guide, copies of which are enclosed. We urge that it be read 
very carefully by interested high school principals. The assessment study design we are using calls 
for ten schools to use CPSS and ten to carry on with services as planned for the 1978-79 school 
year. Essentrally, we are substantiating that schools using CPSS have improved planning for career 
guidance programs significantly more than those who do not use CPSS. The objectives for the 
assessment of CPSS are noted in Attachment 1. 

It is important that the schools using CPSS and those not using it be fairly comparable Two 
common characteristics are ( 1) that all schools participating in the study must be motivated and 
committed to improving comprehensive career guidance services to students, and (2) that the 
schools must be comparable on criteria such as stability of school staff, size of the guidance staff 
number of students who enter college, drop-out rates, and socioeconomic status. Once a set of 
schools with these attributes is identified, 20 schools will be assigned randomly to user and non- 
user groups, ten schools in each. To protect the process of me study, user and non-user schools 
should not be in communication with each other regarding the study. For now, we need many 
schools volunteering who are motivated and committed to improving comprehensive career guid- 
ance services for students, and who meet the criteria for comparability. This is volunteering with 
the understanding that, if selected, only one school from your district could be assigned randomly 
to a user group and one to a non-user group. Enclosed are copies of a School information Checklist 
which should be distributed to and completed by those high schools (three or more) in your district 
which may wish to participate in the assessment study. The completion and return of the School 
Information Checklist by a school's executive will be considered an application to participate in the 
study. Each applicant will be notified in writing of the outcome of the final selection. 

To defray costs of participation in this study, monies will be set aside for each of the 20 
schools involved Those actually using CPSS will have a budget planned for approximately $5 000 
while those in the control group will have a budget planned for approximately $500 A written 
agreement will be executed between each school and the National Center 
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Tlie schools selected to use the product will be expected to commit a person to lead the 
implementation of CPSS and act as a liaison person with the National Center. This person will 
average at least a half-time schedule with CPSS, plus a training session at the National Center, 
Other school personnel will need to participate in the study but not as much time will be involved. 
User schools also will supply pre- and post^test data through data collection instruments provided 
by the National Center. A graphic description of a user school's role in this study is in Attachment 
2, Design Structure for the Assessment of CPSS. 

Similarly, those schools not using CPSS must also have a liaison person who, essentially, will 
supply information to National Center staff. This person will likely spend the equivalent of 10-15 
days during the school year on data gathering and reporting. No Lime other than survey response 
(pre- and post-test data) will be needed from other staff. 

We invite you to participate in the selection of schools once you have ascertained full commit- 
ment to such a study. To answer any questions you may have about the study, a member of the 
project staff will telephone within the next ten days. Otherwise, direct any comments or questions 
to James PearsoL Program Associate, (614) 486-3655, x354. The deadline for applications is 
August 21, 1978. 

We shall appreciate your thoughtful consideration and look forward to further communication 
with you regarding the proposed assessment study. 



Sincerely, 



Robert E. Taylor 
Executive Director 




RET/yy 



Enclosures 
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The Ohio State University • 1960 Kenny Road ♦ Columbus. Ohio 43210 
Tel; (614) 486 3655 Cable CTVOCEDOSU/Columbus. Ohio 



July 20, 1978 



Dear Principal: 

We at the National Center for Research in Vocational 
Education are interested in identifying a set of high schools 
to participate in a controlled assessment study during the 
1978-79 school year. The product to be tested is the Career 
Planning Support System (CPSS) , a systems approach to upgrad- 
ing career guidance services. CPSS is a comprehensive career 
guidance development tool that emphasizes program planning, 
implementation and evaluation. A further description of CPSS 
will be found in the CPSS Decision Guide which accompanies 
this letter. 

To identify those schools which might participate in the 
study, we have asked superintendents in several large cities 
throughout the United States to recommend those high schools 
in their districts which might be motivated to improve career 
programs and which are comparable among several variables 
such as size and socioeconomic status. Once interested 
schools have been identified, tern schools will be randomly 
assigned to a CPSS user group and ten wilL be randomly 
assigned to a non-user group (a control group) for the study. 

The attached School Information Checklist is a means 
whereby we may select schools to participate in the study. 
If you would like to participate, please return the Checklist 
in the preaddressed envelope provided. 

We shall appreciate your completion of the attached 
Checklist and look forward to your participation in the site 
selection process. To answer any questions you may have, a 
member ot the project staff will telephone your office within 
the next week. 



Sincerely , 




jel 
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CPSS Assessment Study 
School Information Checklist 



Directions 

We are interested in gathering information about your 
school for the Career Planning Support System (CPSS) assessment 
study. The attached checklist is a simple method to collect 
certain information about your school. Most, if not all, of 
the information requested may be obtained from school records, 
for example, average daily attendance. When completing the 
form, please approximate on those items where you lack suffi- 
cient information . 

Please complete items one through twenty-one (1-21) as 
presented. If any of these items (1-21) require clarification 
or are not applicable, please refer to item 22. Please include 
any comments in item 22. Thank you for your assistance. 




CPSS Assessment Study 
School Information Checklist 

High School: 



Zip Code 

Area Code Phone Number 



Principal 



Number of years in position 



The average daily attendance during the 1977-78 school 
year: 



Has the average daily attendance increased or decreased 
by more than ten percent during the past two years? 

Yes No • If yes , please specify reasons : 



The number of f cculty/staf f in the high school during 
the 1977-78 school year: 

teachers counselors administrators 



What are the averaae years of total experience for 
f aculty/stctf f ? 



The racial/ethnic composition of the high school's stu- 
dents during the 1977-78 school year: 

a. percentage of Black students 



b. percentage of Native American students 



c. percentage of students with Asian ancestry 



d. percentage of students with Spanish surnames 

e. percentage of other non-White students 

f . percentage of White students 

Percentage of students in various program (curriculum) 
areas : 

college Prep Special 



General ^. , 

Other 

Vocational 



(Specify "other" ) 



What is the dropout rate for your high school? 



Percentage of students who graduated in June 1977 and 
attended college: 



Turnover rate among faculty at the school : 
% per year 

Is the school considered a leader in the adoption of 
educational innovations? (Typical indications that a 
school is a leader are that it has received state or 
federal funds for more than one pro j ect during the last 
three years or that the school has participated in 
field tests of product development.) 



Yes No. If yes, briefly describe the educa- 
tional innovations that have been adopted during the 
past three years. 



How many feeder schools does your school have? 



What is the mean test score (s) on the SAT or ACT for 
juniors and seniors in your school? If available, use 
the 1977-78 score reports. If not, use the 1976-77 
score reports. 

76-77 77-78 SAT V M (Please refer 

to SAT summary report for your school and check iV) 
appropriate year of the report.) 

76-77 ^ 77-78 ACT Composite (Please refer 

to ACT High School Profile Report for your school and 
check (n/) appropriate year of the report.) 

What is the total population of the attendance area for 
your school? 



What is the estimated average income within your atten- 
dance area? (check one) 

$ 5,000 - 10,000 

$10,000 - 15,000 



$15,000 - 20,000 
$20,000 - 25,000 
$25,000 - above 



What grade levels does your school have? 9-12 

or 10-12 

Does your school district or state have a comprehensive 
career guidance program model? 

Yes No 



If yes, is your school required to use the career guid- 
ance program model? 

Yes No 



Please list three unique aspects of your school: 



22, Items you may wish to clarify or comments: 



3 
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APPENDIX J 
Work Statement 



mUK STATEMENT FROM A PURCHASE OF SERVICE AGREEMENT 
WITH A COOPERATING SCHOOL DISTRICT 



The contents of this work statement include the scope of services 
provided by The National Center for Research in Vocational Education 

t-he Jefferson County Public .Schools for the Assessment of the ^ 

Career Planning Support System. The effective dates for this proposed effort 
are September i5 , 1978 t.o June 30, 1979. Amendments to this agreement may be 
effected through mutual consent in writing by all the signatories affected by 
the changes. 
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1^0.0. The National Center for Research in Vocacioaal Education will pro- 
vide niie oxaceriais, resources, and services listed in 1-1-0- through 
1.6,0. 

1.1. 0, Two conplete sets of the CPSS materials (Coordinator's Handbooks, 
eight guides, four audiovisual presentations and one package of 
camera-ready masters). One set will be used by the experimental 
school for Che purpose of the assessment study. The other set 
x/ill be delivered to the control school once the assessment study 
Is completed. 

1.2.0. Three days of pre-service preparation at the National Center for 

Research in Vocational Education for the experimental school coor- 
dinator to familiarize him/her with the CPSS product and to enable 
him/her to accomplish the evaluation tasks associated with the 
assessment study. 

1,3. 0, Technical assistance to the experimental school coordinator as need- 
ed, as requested, or as determined by The Mational Center staff vith 
the identification and resolution of problems associated with the 
assessment study and the use of the CPSS -product. This assistance 
from the National Canter staff will be provided to the site via 
telephone, mail or travel to the site depending on which method 
is deemed most appropriate by the National Center's staff. 

1.4.0. National Center staff to conduct on-site monitoring and progress 
assessments and knowledge and attitude posttests, 

1,5,0. Sufficient copies of status report forms and knowledge and attitude 
tests. 

1,6,0. Costs reimbursable effort is not to exceed $5,500. Five thousand 

dollars will be set aside for the experimental school and $500 will 
be set aside for the control school (refer to budget, attachment A) • 
The payment schedule will b-^^ in two ins tallmencs ♦ The first payment 
dace will be January 15, 19/9 in an amount for which invoices have 
been received. The second payment date will be June 30, 1979 for the 
remaining invoices. 

2.0.0. The Jefferson County Public Schools will provide the materials, 
resources , and services listed in 2.1.0. through 2.5.2., and will 
conform to the provisions incorporated in HEW Standard Form 315 
(incorporated by reference only) . 
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2.1.0. 
2.2.0. 

2.3.0. 
2.3.1. 

2, J. 2. 



Monthly, one-page reports due on the first of each month (beginning 
with December 1, 1978) to include invoices and descriptions of ex- 
penses percaining to the CPSS Assessment Study. 

One central administration staff member (not a staff member at either 
the experimental or control school) to adminiscer the knowledge and 
attitude pretests to the CPSS Steering Committee at the experimeacal 
school. 

■ E:cr)erlmen tal School 

One permanent staff member released one--half time to coordinate CPSS 
in the school and to provide evaluative data for the assessment study. 
It is understood that this will be a regular assignment and uot extra 
duty. 

Resources and services to include: 

a. Printing (offset, etc.). The total number of pages should not 
exceed 8.5 times the total number of students in the school. 

b. Duplication services (electrostatic, etc.). The total number 
of pages should not exceed two hundred (200). 

c. Office supplies (excluding paper for printing and duplication). 
Supplies should include two reams of letterhead, 1,400 business- 
size envelopes imprinted with the name of the school, and postage, 
if the questionnaires are mailed. 

d. Equipment. A filmstrip projector and a cassette tape recorder. 

e- Telephone Service- Long distance telephone calls to the National 
Center as needed. 

Time of r :ofessional staff, in addition to the school coordinator, 
as follows: 

a. Five to seven faculty/staff, who will serve on the CPSS Steering 
Committee an average of one and one-half hours per week during 
the academic year, and who will complete 1 1/2 hours of pre and 
posttests. 

b. No more than 16-20 professional staff members who will serve on 
the Needs and Resources Assessment Task Forces at an average of 
two hours per week for an average of 8-10 weeks each. 

c. Other professional staff as needed to implement CPSS in the 



school. 
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2.3.4. Primary outputs, to include demographic data about school and com- 
munity, documentation of actual and potential career r'.eveiopment 
program resources, statements of program goals, stateiLents of be- 
havioral objectives^ proposals for career development activities, 
and reports of evaluations of career development activities. One 
copy of each output and any changes to that output will be pro- 
vided to the National Center staff on specified deliverable dates. 

2.3.5. Secondary outputs to include a report of career development pro- 
gram planning, pre and posttest data, and project logs. 

2.3.6. The school may publish and make available ir.iormation concerning 
its participation in the assessment study on ce the assessment 
studv is completed . A copy of each article submitted by the 
school shall be sent to the National Center after its publication 
or presentation- 

2«4.0. Control School 

2. 4. J.. One permanent staff membe"" released for the equiv^^lent of 10 to 15 
days during the school year to gather data and complete reports. 

Z.4.2. Postage and Jong distance telephone calls to the National Center as 
needed ♦ 

2.4.3. The outputs shall be a reporu of career development program plan- 
ning, both pre and posttest. 

2. 5* 0. Deliverables 



2.5.1. Experimental an d Control Schools 

A. Due Dates: 9/25/78-11/15/78 Career Development Program Status 
Report 

1. One Status Report to be completed by school personnel and 
delivered to Center project staff during Fall on-site 
visit at the school (a two-day visit to be arranged be- 
tween 9/25/78 and 11/15/78 — two weeks notice will be 
provided) . 

2. Due Dates: 5/20/ 79 - 6/30/79. One Status P.eport to be 
completed by school personnel and delivered to Center pro— 
ject staff during Spring on-site visit at the school (a 
two-day visit to be arranged between 5/20/79 and 6/30/79 — 
two weeks notice will be provided). 

2.5.2 Experimental Schools 

A. CPSS Coordinator Training - Three days training provided by the 
National Center staff tor the CPSS Coordinator, to familiarize 
him/her with the CPSS product and project deliverables Cdates 
Co be arranged prior to utilization of CPSS in the school). 
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KnowLea-^e and Acticucie Assessmencs 

1. Duti Dace: 12/15/78 - to be adminisCered by non-school per- 
sonnel to the Steering Committee faculty/staff at the first 
committee meeting. 

2. Due Dates: 5/20/79 - 6/30/79 - to be administered to the 
Steering Committee faculty/staff by Center project staff 
during Spring on-site visit at the school (a cwo^day 
visit to be arranged between 5/20/79 and 6/30/79 ~ two 
weeks notice will be provided) - 

C. Results of Needs Assessment - Due Date: 3/1/79 — Tables AO, 
41, 42, 43, 44, 45, 46, 47, 48 (see page 25 - Coordinator's 
Handbook, "Results") - 

D. Results of Resource Assessment - Due Date: 3/1/79 - Tables 
34, 35, 36, 37, 38 and 39 (see page 6, Coordinator's Handbook )* 

E. Goals Data - Due Date: 3/31/79 - Tables 50, 51, 52 and 53 
(see page 35, Coordinator's Handbook , "Results") - 

F- Caree r Development Unit (CPU) Data - Due Dates: 5/20/79 - 

6/30/79 - Copies of at least two CDUs developed and evaluated 
as a result of CPSS involvement. Form No. 21 and Tables 54 
and 55 (see pages 52 and 57, Coordinator's Handbook ), Copies 
to be delivered to Center project staff .during Spring on-site 
visit at the school (a two-day visit to be arranged between 
5/20/79 and 6/30/79 - two weeks notice will be provided). 

Project Log - Due Dates: Due every two weeks beginning Novem- 
ber 15, 1978 and ending June 15, 1979- The logs are brief 
progress reports and are mailed directly to National Center 
staff. 




Attachment A 

ASSESSMENT OF THE CAREER PLANNING SUPPORT SYSTEM 

Proposed Budget 
September 15, L978 - June 30, 1979 

The following budget depicts the budget areas, in accordance with al- 
lowable costs, within which the Jefferg^on Counr v Pttblir. Srhnoig will re- 
quest reimbursements 

I. Personnel - (This area refers to costs incurred for the time of 
permanent professional staff participating in the CPSS Assess- 
ment Study.) Items listed in this section may include costs for 
stipends, extra duty subs titute teachers, fringe benefits and 
partial salaries. 

Item Estimated Budget 

a. Stipends and extra nav $5273 , 00 

b. 



Description! 

Stipends for steering committee members, needs 
assessment and co ordination. 

II. Supplies - (This area refers to costs incurred through the purchase 
of supplies and telephone service for the CPSS Assessment Study, 
see 2.3.2. and 2.4.2. of agreement). 



Item 


Estimated Budget 


a, Sui.'iOlio.s 


$U7. 00 


b. Telenhono 


$Rn. 00 


c. Stamps 


$1 ^o.nn 




Description: 


$227.00 



N[OTE: teams III, IV and V require specific requests for permission and 
prior approval before funds will be allocaced. 



III. Equipment - (This area refers t:o coses incurred through the purchase 
of equipment (direct and indirect costs) for participation in the 
CPSS Assessment Study) . 

Item Estimated Budget 



a. 
b. 



Description: 



IV. Travel - (This area refers to travel costs, if any,,. Incurred by 
school staff participating la the CPSS Assessment Study) . 

Item Estimated Budget 



Description: 




Other Costs - (This area refers to other coses incurred by the 
school or school staff as a direct result of participation in 
the CPSS Assessment Study, i.e., expenses for a retreat for the 
CPSS Steering Committee), 

Item Estimated Budget 

a. 

b. ■ 

c. 

d. 



Description: 



ASSESSMENT OF THE CAREER PLANNING 
SUPPORT SYSTEM 



Jefferson County Public Schoo3-S 
(School District) 

Van Hoose Education Center 



(Address) 
3332 Newburp: Road 



Louisville, Kentucky 40218 




Date: 12-1-78 



(Signat:ur<^) 

u ^ 

E,C , Grayson 

(Typed Name) 

Superintendent of Schools 
(Title) 



7 



APPENDIX K 
Joint Dissemination Review Panel Report 



Copy Submitted to NIE 
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PROGRAM AREA: Career Development Progrnm Planning 

I. TITLE OF PRODUCT: The Career Planning Support System (CPSS) 



II. DEVELOPER: The National Center for Research in Vocational Education 
III. FUNDING: National Institute of Education. Testing $340,373 
IV. DEVELOPMENT AND TESTING PERIOD: Development: 1971-1976; Testing: 1978-1979 
V. HRIEF DESCRIPTTCjN OF CPSS: 

Amoiif, Lhe prit)ritit^s identified by the career cducat Jon movement of the seventies 
wc.»rc 1) i\ need to blend student <:areer development into the mainstream of educational 
practiro, ant! 2) a need to meet increased accountability demands in the delivery of 
inst rut: L ional and counseling services in public schools. To meet these two needs 
recent research activities have emphasized the importance of systems methodology in 
properly planning, implementing, and evaluating career development programs 
ICampbell, 1975; Campbell et al. , 1971; Hosford and Ryan, 1970). 

Mitchell and Gysbers (1979) reported that an emerging direction for career de- 
velopment and guidance in schools is the guidance system comprised of a series of 
interrelated planning, design, implementation, and evaluation components. Herr 
(1979) recommended that guidance at the local school level be based on student needs 
and planned as a total program with goals, objectives, activities, and student 
outcomes. A National Vocational Guidance Association Posirion Paper on Criteria 
for Career Guidance Programs (1979) stated, *'in order to achievp. lasting effective- 
ness. It Is important that (career development) program planners follow a compre- 
h.ensive student needs-based and evaluation-oriented approach to program development." 

In response to the need for systematic program planning for student career 
development programs, tiie National Center for Research in Vocational Education 
developed and tested CPSS from 1971 to 1973. A two-year (197A-76) field test of 
CPSS resulteil in important revisions of the materials. Thir '*_y-eight individual 
hi^^h scliools, ranging from rural scliools of less than 100 to large urban and 
suburban schools of more than 2,000 students, participated in the field testing. 
This ^submission is based on data from an assessment study conducted in 1978 and 1979. 
The 1978-79 assessment of CPSS involved eighteen high schools in seven states. 
Tlu' purpt^se of the assessment study was to test the effectiveness of the 
CPSS materials as a high sc.liool career development program support system. 

CrsS consists of handbooks, reproducible forms and f ilmstrips that describe a 
K'omprehens i ve organ i J'.a t iona I framework and procedural steps a school staff can use 
to create an accountable, school-wide high school career development program. The 
following list describes the complete set ^of CPSS materials: 

e The Coordinator's Training Guide is a self -instruct ional training guide 
for the part-time CPSS coordinator. 

e The Coordinator ' s Handbook contains instructions that describe step-by-step 
procedures for managing and implementing CPSS in the high school. 

• Camera -Ready Forms are reproducible copies of each form needed for the 
questionnaires, instructions, CPSS Program Information File, etc. 

o Handbooks 

The Advisory Committee Handbook defines the responsibilities and duties 
of Advisory Committee members (five copies). 
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Asse ssing Resources guides a resource leader in directing a task force to 
"colTect information on and account for the use of resources in the school 
and community. 

Assessing Needs; Surveying provides instruction for preparing, adminis- 
tering, and collecting survey questionnaires for students, graduates , 
parents, ami fncu I ty /staf f (five copies)* 

A ssessing Nee dsj l\ abulation contains instruction on manually tabulating 

data collected by questionnaires (five copies). 

A nalyzing Meth ods directs a methods specialist about the availability and 
application or"^""giadance methods and how to integrate this knowledge during 
the construction and review of career development units. 

The Manual for Writin g; Behavioral Objectives is a self-instructional 
resource for a beliavioral objectives specialist. 

Writing Behavioral Objectives informs the behavioral objectives specialist 
about the function of behavioral objectives in the construction of career 
development uni ts . 

Produci ng Caree r Development Units (CPUs) provides direction for develop- 
ing career guidance/development activities. 

Q I'i Lmst rip/Audio Tape Presentations include: 

AV-l: "An OrientaLxon to CPSS" — orients interested persons to CPSS. 

AV-2: "Shaping Program Goals" — gives an overview of how the needs and 
resources assensments lead to goals for a school. 

AV-'J: "Behavioral Objectives" — accompanies the behavioral objectives 
manual . 

AV-4: "Producing CDUs" — gives an overview of the career development unit 
process . 

Claims of effect iveness . CPSS is intended as a set of tools to assist with 
institutionlTl changes in planning for career development programs in high schools. 
It is assumed that the school staff using CPSS is motivated to plan for the school's 
career development program. The main claim of this submission is stated below. 

Use of the CPSS mrtterials for one academic year enables a high 
school staff to implement a systematic planning process for student 
career development ])ro^', rams. 

I'^^r clie purposes of this submission "systematic planning process" includes the 
f o I lowing elements : 

• listahLishment of an organiicational structure facilitating a career develop- 
ment program, to include clearly designated leadership and permanent active 
committees and work groups. 

e Assessment of the career development needs of local students and use of the 
results of the needs assessment in the career development program. 

• (Creation of explicit career develo|)ment goals reflecting assessed student 
caret^r development needs, listed in order of importance, 

• Creation of student behavioral obj ec t Ives designed to implement the goals. 

• Creation of stude n t activities to achieve the objectives and goals. 

Career development In the CPi-S perspective is defined as the process by which 
an Individual student .acquires the basic, non-technical skills needed for functioning 



in the world of work, A career development: program is a sequence of activities 
designed to help foster student career development. 

Intended users of CPSS . High school personnel and students cooperate in use 
of CPSST 

Costs to schools . Table 1 shows cost estimates for using CPSS during the first 
year and subsequent years. The figures could be converted to costs per learner by 
dividing by the number of student users, but this ratio does not seem like a useful 
statistic since CPSS is designed to affect directly the institution, and the main 
claim of this submission refers to institutional change, not learner change. 
Because costs may vary among schools,, a range is entered in the table. 

TABLE 1. COST ESTIMATES PER SCHOOL 



Personnel 
Staff Training 
Special facilities 
Equipment 
Consumables 
Other costs 
TOTAL COSTS 



Eirst Year 
(Nonrecurring Costs) 

2900~7250 
0 
0 
0 
123 
2 60 
3283-7633 



Subsequent Years 
(Recurring Costs) 

2175 — 2900 
0 
0 
0 
61 
60 

2296—3021 



VI. EVIDENCE OF EFFECTIVENESS 

Design of the field test . Data supporting the claim for effectiveness were 
gathered, using a pre-post, experimental-control group design, on 18 high schools. 
The high schools were located in Arizona, Maryland, Illinois, Kentucky, Tennessee, 
Florida, and Colorado. Table 2 displays descriptive statistics for the test sites 
Ten of the 18 participating schools used CPSS for one academic year, and the re- 
maining eight did not. In this document CPSS users frequently are referenced as 
experimental schools and nonusers are termed control schools. Measurements on all 
variables related to the main claim were taken before and after the school year in 
which experimental schools used CPSS. 

TABLE 2. CHAEACTERISTICS OF TEST SITES 





Control 


Experimental 




Average of Ciia ractteristic 


Schools 


Schools 


t -value 


Size of student population 


1916 


1943 


.074 


Ratio of facultv & staff to student pop. 


19.49 


17.64 


1.540 


ACT/SAT scores*' 


15.67 


16.84 


. 748 


Estimated family income 


$12000 


$13125 


.607 


Drop-out rate 


7 . '4/o 


10. 0% 


1.375 ' 


Percent white 


38% 


51% 


.814 



NOTE: Tablv entries are averages over the control or experimental schools, as 

labeted- Experimental school refers to a school that used CPSS during the 
study, and control school refers to a school that did not use CPSS. 

*Five schools made SAT scores available, and the remaining 13 submitted ACT 
averages. The five SAT scores were converted to the metric of ACT by 
dividing them by the ratio of the average over schools SAT to the average 
ACT. 
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Kach sc-.lu)ol pruvidtMl a ;)a r t-L imo coordinator who was responsible Cor the pre- 
paration and completion of data colloction forms and who served as the on-site 
contact person with the National Center staff. In the experimental schools this 
contact person also served as the CPSS coordinator. The experimental school coor- 
dinators received a three day training in CPSS procedures in November 1978. The 
training was conducted at the National Center by project staff. Training normally 
is not necessary for use of CPSS; it was provided in this instance to help accelerate 
the normal process of creating a career planning system, in order to complete the 
study within the specified time period. 

Hoth experimental and control schools were monitored by monthly telephone calls 
and one wite visit in February, 1979. This was in addition to pretest and posttest 
site visits to all schools in November or December of 1978 and May or June of 1979. 
The telephone calls and site visits included very little technical assistance. 
Experimental school coordinators were requested to complete project logs twice a 
month, closer iblnft the progress of CPSS in the school. 

Participating schools volunteered in response to a national publicity campaign. 
The original intent was to assign participating schools at random to experimental 
and control conditions, but due to insistence of local school administrators random 
assignment occured in only four instances. In the remaining cases, local school 
officials made the determination. Experimental schools were paid 5000 dollars to 
defray expenses, mostly to pay for personnel time. Control schools were paid 500 
dollars and given a set of CPSS materials at the end of the study. 

Self selection of schools into the study at first appears to threaten the ex- 
ternal validity of the results, but, on reflection, probably poses no such threat. 
Users of CPSS certainly will all be self-selected; therefore, the sample is drawn 
from the universe of probable users. Inability to control assignment of schools to 
experimental and control conditions poses some threat to the internal validity of 
the design. The pre-post nature of the design, equivalence of the experimental 
and control schools on key variables (see Table 2), and the magnitude of the gains 
for experimental schools suggest that the results are not likely due solely to 
nonrandom assignment, however. The main threat to the internal validity of the 
study is tl:c interaction between those selected into the experimental group and 
"maturation" (i.e., changes that would occur without the treatment, but only in 
experimental scliools) (Campbell and Stanley, 1966). While interaction between 
"maturation" and selection cannot be entirely ruled out as a contributing factor 
in experimental school gains, the gains reported below are too large reasonably to 
be attributed solely to the interaction of maturation and the treatment variable. 
These gains are all over one standard deviation. 

Me/t surement . Two data collection forms, the Career Development Program Status 
Report and the Verification Checklist, and one rating instrument, the Career De- 
velopment Program Rating Ins^trument, were developed and used for the study. The 
Status Report and Verification Checklist were used to collect iS!&€ia3IiaJ&=^1cSn from the 
field sites. The information was then reviewed and rated by a fifteen member review 
panel with acknowledged expertise in career development related areas. The review 
panel members indlviduc^ lly answered questions on the Rating Instrument by referring 
to Information collected on the Status Report and Verification Checklist for each 
school. All aucilyses reported in this submission were based on data drawn from 
the Rating Instrument. 

The Career Development Program Status Report and the Verification Checklist 
were developed by project staff. A review of the forms by external consultants 

4 
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IndlcaLed tlwiL Llic fornus priwltle infornuit ion reLated to the elements of a system- 
atic ctireer deveiopment program and Ikwc (U)nLent: vaiidlLy. 

The Status Report was completed by school personnel in all schools who docu- 
mented the extent to which their existing career development program planning 
reflected the basic components of systematic career development program planning. 
These data were collected before experimental school coordinators were trained. 
The completed Status Report was reviewed on-site by project staff and missing 
data were obtained. Examples of the type of information collected through the 
Status Report include data about career-education goals, assessment and evalua- 
tion related to career education, and student career-development activities. 
Career development activities include^ but are not limited to, curriculum units, 
visits to local businesses, and career days. 

The Verification Ciiecklist provided a means by which project staff could cor- 
roborate> clarify, and expand the information recorded on a school's Career Develop- 
ment Program Status Report. During the pretest and posttest site visits, a National 
Center staff member completed the checklist with school personnel assistance, and 
both persons signed the completed form indicating agreement on the accuracy of 
the information. Examples of information gathered on the Verification Checklist 
Includt* data about career^oducat ion needs, career-education goals., and committee 
organ i //Jt ion related to thu career education of students. 

The Rating; Instrument was developed by project staff with the assistance of 
an L'xiernal instrument design specialist. Two factors basic to the design of the 
rating instrument were: (1) inclusion of items that were clearly answerable given 
the descriptive information that was being rated, and (2) the exclusion of items 
that did not allow control schools a fair "opportunity to receive a high rating. 

The Rating Instrument is divided conceptually into two major parts. Part One 
asks questions concerning specific facts describing the school's career development 
program planning. Detailed questions are asked about the conduct of needs assess- 
ment, goal formation, objective writing, student activities, and organizational 
structure. Part Two contains six summary questions asking raters to form broad 
Judgments concerning each of the five elements of a systemati'^ planning process 
for career development listed on page J. of this submission. The sixth 
question in Part Two requests a judgincnt regarding the overall quality of the 
eareer development program. Part One of the Rating Instrument, thus, is designed 
to familiarize tlioroughly e^ich expert rater with the facts, in preparation for the 
judgments requested in I'art Two. 

A group of fifteen eminent persons in fields related to career development 
resecirch and practice was assembled at the National Center to assist with inter- 
pretation of the information collected from the field sites. Panelists completed 
two twenty-one page rating instruments for each participating high school. The 
first completion provided a description of all schools at the beginning of the 
school year, 1978-79, and the second completion described the career development 
program in all schools at the end of the school year. During the year the experi- 
mental schools used CPSS materials and the control schools did not. It should be 
noted that all identifying information, e.g., state, city, school, name, address, 
and dates had been removed from the data sources prior to the ratings. 

At least three panelists were assigned at random to rate each experimental and 
control school. Assigning more than one rater to each school permits numerical 
assessment of reliability of the ratings and yields more accurate results than 
rould be obtained from a single rating per school. Pretest and ])osttest ratings 
lor cat^h schoc:)! were done by the same group of panelists. Panelists were given 

ERIC 1 ' ' ^ 



no informacion about the nature of the design prior to the rating session. In 
particular, experimental and control schools and the pre-post feature of the 
design were not identified to panelists. In a final debriefing session, after 
all rating activities had been completed, the panelists were told that they had 
participated in an assessment study of the Career Planning Support System, They 
were given copies of CPSS materials, a study abstract, and informed of all aspects 
of thu study. The panelists indicated that they had neither surmised the nature 
of thv' study nor recognized that they had rated pre and posttest data from the 
same si hoi)] s . 

Thf main reason tor use ol a paneJ. of Judges is reJated to the nature of the 
subjetM matter, Kew people would doubt that efficient organization and planning 
comprise important aspects of high school career development programs. Yet the 
important features of efficient organization and planning remain uncodified in 
sufficient detail to permit completely objective measurement. In such instances, 
human judgments are essential. Hence, a panel of individuals was assembled with 
the exporienc:e, training, and reputation to provide the most accurate judgments 
avail able. 

Because of thf»ir importance to the presentation, the six questions addressed 
by the panelists are reproduced verbatim below. 

1. Estimate the extent to which the school staff was organized to plan system- 
atically a comprehensive career development program by evidence of clearly 
designated leadership; administrative cooperation; and permanent, active 
groups and committees. 

2. Kst imatc the extent to which a student career development needs assessment 
was eonduttted, tabulated, properly interpreted, and the data utilized for 
planning the career development program, 

J. Kstimate tlie extent to which a comprehensive set of ordered career develop- 
ment goals reflecting assessed student career development needs were 
developed and used in planning, implementation and evaluation of the program. 

A. Irlstimate the extent to which a set of behavioral objectives was developed 
reflecting specific goals and containing a clear statement of the intended 
audlL^nce, behavior, situation and standard of mastery. 

5. Kstimate the extent to which career development activities were developed 
that reflect student needs, goals, and associated objectives, and that 
indicate methods, target student group and outcome measures by referring 
tc^ the two attached career development activities. 

Ikised on the available information (including all career development acti- 
viti(»s), rnt o tlie overall quality of the school's career development 
p r ( kj."; ram , 

To au.swt»r these ({iiest i i>ns, raters referred to all information on the Status 
Ki-poi t and VtT it i L-at it)M Olie^klist from each schoo.l . Thus, raters had at their 
disposal data regarding schools' student career development needs and goals, 
career <iL»ve I opment activities designed for use with students, and organization 
of career-development program planning. Ratings for the first five of these 
Items were recordetl on a five point scale ranging from "limited extent" (scored 
0) to "greaL extent" (scored A). Ratings on the overall quality were also re- 
corileil on a five lurlnt scale ranging from zero to four, but the two extreme 
points w<*re lahelcn! "very low <iuaiity" and "very iii>;h quality." 
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'I'lit/ MiiiL <»1 analysis Inr .ill s(ali.slic:al r'tsulLs is the scliuol, A sc:ure des- 
rribiu^; i-acli si-liool <ni each variable was calculated by rorming the average over the 
three or four raters who rated each school. Agreement among raters for a given 
school, thus. Indicates the reliability of tht? scores, and, conversely, disagreement 
among raters indicates unreliability. The discrepancies among raters of a given 
school can be compared to differences in average ratings across schools. This 
basic idea Forms the conceptual basis For calculating reliability coefficients 
based on an analysis of variance model (see Winer, i97l: 283 ff). The idea is to 
compare a mean-square within schools to the mean-square between schools. Since the 
t>l>lo( t td the design is id minimize pretest differences among schools, these calcu- 
Liiions .lit' based on posLLest scores only. This procedure is quite analogous to 
ca l<'ii laL ion ol retiability c oe I f i c 1 lmu: s fruui student scores on a test following a 
curriculmn tuill, because a "flixjr" effect a rt i f i <: i a J 1 y dellales reliability calcu- 
lat Luna derived I rom pretest siujres. The point is, that there is very little 
variance between schools on the pretest; all scliool.s score low. The calculations 
omit consideration of "anchor points" (Winer, 1^71: 289 ff), thus yielding somewhat 
conservative estimates of reliability. The formula used approximates an unbiased 
estimate of reliability, assuming no anchor point differences among raters (unlike 
correlational methods such as split half or coefficient alpha, which are biased 
dov/nwartl) . 



Reliability of these items is unifomiiy quite high, 
from .829 to .932, and average .881 (see Figure 1). 



The numerical values range 



Piinel 1: Organization 
p ^ •001 



1+ 
0 



re 1 iab L I i ty = . 852 

average 
eon f i denc:e 
rat ing = 87.0 



Panel 2: Needs 
^, p < .001 
2 
1 

OL 



re 1 lability = . 922 

average 
ecmf idenc e 
rating = 91. 9 



Panel 4 : Objectives 

4- 

p < .001 

2 
I 

0 




reliability = .900 

average 
conf idence 
rating = 90.7 



Panel 5 : Activities 
4^. 

p < .001 

2.. 
I- 
0~ 



reliability 

average 
conf idence 
rating = 87.9 



.850 



Panel ); (loals 

4- 

p V .001 

2 
1 

0 



re I iabil ity = . 932 

average 
confidence 
rating = 91.5 



anel 6: 


4- 




3 


P 


2. 




1 • 




0 


' f- 



reliability = .829 

average 
confidence 
rating = 87 . 7 



FICURK 1. ANCOVAs, REL I AB [LITV , A^JD CONF I DENCF. RATINGS FOR SIX SUMMARY MEASURES. 
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1,1 .mMIiIou rc) ri» I i al) i 1 i L y <*)*M' I i c i tMiUs b.'i.siul on nKr^'tMiirnl anumK differont 
raters ot I lie saint* .srliools, panel isl^ woro asked to t.'sl Lmat e their confidonce in 
each rating Lliey made, Tlie eonfidenre rating was the same for oach question. 
Raters wtjre asked to lAaa^ a -rdieck along a scale from zero to .100 indicating their 
judgments regarding the likelihood that their answers were accurate. The format 
of the confidence rating is reproduced below. 

CONFIDENCE ElMTNO: Please estimate the"" p'robability that your answer is correct 
by placing a check at the appropriate location on the scale. 

j I J J J I I L \ I i 

0^ To 20 30 "^40 50 60 70 80 90 100% 

no confidence ccnnp^lete confidejice 



Tlu' aveiage c on I i deiii-e ratings ul panelists Is quite high, ranging from 87.0 
to purLteni , thus reinioroing the reliability calculations. In spite of the 

need tor approximate judgments, therefore, it is concluded that available evidence 
is eonsistent with the view that the measurements are accurate to within tolerable 
1 imlts . 

Data analysis method^; . The major hypothesis in this study is that school 
staffs using the Career Planning Support System will produce greater change toward 
implementing a systematic planning process for career development programming than 
will school staffs not usinj; CPSS. The statistical method for assessing this hypothesis 
is analysis of covariance (ANCOVA) , The dependent variable:'- for the ANCOVA are 
posttest scores de^;cribing the planning process of each school at the end of the 
experiment. There are two independent variables including one categorical factor — 
experimental condition defined by the use or nonuse of CPSS — and one covariate 
defined as the pretest score corresponding to the posttest dependent variable. 
Conceptually, the ANCOVAs describe differences in posttest scores between schools 
using CPSS and schools not using CPSS, under statistical control for the pretest 
score, 

Altliuugh it does not appear to be widely recognized, the ANCOVA model can be 
viewed as a model of cliange. Conceptually, the ANCOVA can be viewed as expressing 
tlie following hypotliesis: Change over the period of the experiment is greater for 
schools using CPSS tliaii for schools not using CPSS, when statistical controls for 
the effect of the starting iioint (pretest scores) are applied. 

Results. The major results of the study are summarized in Figure 1. Each of 
the fTrst fTve panels of the figure summarize the results for one element used to 
define a systematic career ::. inning process, and the sixth panel summarizes judg- 
ments of the overall quaKty. The panels of the figure are numbered and labeled 
to correspond to the questions reproduced on page 6 of this submission. 

The graphs display plots of mean differences in posttest scores between experi- 
mental schools (E) and control (C) , as adjusted statistically by the analysis of 
covariance for pretest scores on the dependent variable. Alternatively, as noted 
above, these graphs may be interpreted as differences in change from pretest to 
posttest, adjusted for differences in starting point. The vertical axis of :hese 
graphs represent scores on the six items. The horizontal axis does not reflect a 
cont iniuuis srale. Ratlier, the lefthand point (labeled C) corresponds to the control 
group, and t he r ighthand po int corresponds to the experimental group (labeled E) . 
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Tills pii}j i ! li)M iii^ 111 I 'ul C is arl)itrnry, hut w.i.s st» i cct tnl sd lh.it. «i positive slope 
Indlfiiti-s siipp<»rL l<»i \ iu- iii.iin hyjjothesis: Lh:it t*xp<-*r iinenLa 1 sr.hooiti show larger 
tialns whtMi adjusted lor starting point than do control school All six graphs do 
show a substantial positive slope, thereby lending support to the hypotheses. All 
statistical tests are highly significant, with probabilities less than .001. 
(Reported probabilities are for the main effect of the experimental variable, after 
adjustment for the covariate.) 

Whenever random assignment to treatment groups cannot be realized, observed 
(lit* f eront-es between treatment groups, in theory can be due to nontreatment variables. 
The standard methodology tor handling objections of this sort is to introduce some 
type of statistical control for n small group of variables that are likely candi- 
dates to account for observed differences between treatment groups. In the present 
cas«, the sample size is small enough to render such procedures of dubious value. 
One may observe , however , bivariate relationships between selected "control" 
vartabli's and the treatnieiU variable. In the present study the treatment variable 
Is defined by the two categories — used CPSS and did not use CPSS. Averages on the 
following variables were compared statistically for users and nonusers of CPSS: 
student population size, ratio of faculty and staff to students, academic test 
scores, drop-out rate, percentage of the student body who werc^ ninority group 
members, and a rough estimate of family income of the student bndy. As shown in 
Table 2, in none of these five tests were j: tatistically signif i( ant differences 
observed. Hence, it is concluded that the differences between l sers and nonusers 
of CPSS on the six criterion variables are not due to any of these five charac- 
teristics of schools. 

Kducat ional Importance . There are two factors related to th; educational 
importance of the results. First, are the gains of sufficient magnitude to be non- 
trivial? Secondly, does CPSS address an important educational need? 

Widespread application of standard score units renders them useful because of 
implicit standards based on long experience regarding the magnitude or change re- 
quired to indicate substantive importance. Table 3, therefore, presents the results 
shown in Figure 1 given in standard score units. The calculations were carried 
out with the mean and standard deviation of each variable calculated over pretest 
and post test and over experimental and control group3. One might prefer using the 
pretest means and standard deviations because these values more accurately reflect 
tbi general population of schools, the vast majority of which have not used CPSS. 
Reliance on the overall mean and standard deviation show the results in a conserva- 
tive light, lu>wever, since the pretest standard deviation is, for every variable, 
considerably smaller than the overaLl standard deviation. Dividing by the smaller 
standard deviation w( aid magnify differences between experimental and control 
schools. In summary. Table gives each point on the corresponding gr^ph in 
Figure 1 in standard score units, using the overall mean and standard deviation of 
each variable to calculate standard scores. 

T^e results in Table ^ amplify the graphic presentation. The tabulations show 
that ill every instance, after adjusting for pretest scores by analysis of covariance, 
die posttest experimental schools are over one standard dev;Lation 
above the grand mean; whereas, posttest control schools are one-third to three- 
fifths standard devlaticnis below the mean. Effects of this magnitude due to a 
"treatment" are seldom observed in social research. It is concluded, therefore, 
thi'it the !nagnltu<lc of the stan<i;ird scores indicates educationally important gains 
tor the experimental schools. 
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TABLE 3. STANDARD SCORE MEANS FOR ANCOVAs 



Panel 1: Organizational structure 
Did Not Use CPSS « -.38 
Used CPSS » 1.40 




Panel 4: Objectives 

Did Not Use CPSS = 
Used CPSS =» 1.35 


42 


k .-.^el 2: Needs 

Did Not Use CPSS = -.46 
lined CPSS 1,36 






Panel 5: Activities 

Did Not Use CPSS = 
Used CPSS = 1.19 


48 


Panel 3: Hoils 

Did Not Use CPSS = --,62 
Used CPSS - 1.42 






Panel 6: Overall quality 
Did Not Use CPSS = ~. 
Used CPSS = 1.34 


31 



The second aspect of educational significance is ^he need addressed by CPSS, 
As nuted In the opening paragraphs of this submission, the CPSS materials were 
develtjped in response ti) a need for improved career development planning in schools. 
This need has been expressed repeatedly in a variety of professional forums repre- 
senting several professionaJ speciaities. Prior to development of CPSS, a consensus 
developed which reported that systematic planning was an essential ingredient in 
improving career development programs. The CPSS materials are designed to instruct 
school staffs in the use of a systematic planning process and development of 
associated products for building career development programs in high schools. 
The data in this submissicjn demonstrate that tlie materials do enable staffs to 
creiite a systeiuatic planning process. 
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